Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Custom Type Validations #822

Closed
wants to merge 34 commits into from
Closed

Custom Type Validations #822

wants to merge 34 commits into from

Conversation

anish-palakurthi
Copy link
Contributor

@anish-palakurthi anish-palakurthi commented Jul 22, 2024

  • lookahead spec docs for custom type validations
  • significant rework of BAML language server, including changes to the grammar, parser, and IR validation in order to support auto-formatting and type validations.

Copy link

vercel bot commented Jul 22, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Comments Updated (UTC)
baml ✅ Ready (Inspect) Visit Preview 💬 Add feedback Aug 3, 2024 4:29am

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

The pull request introduces new documentation for custom type assertions and checks, along with updates to existing documentation for dynamic types and multimodal input.

  • docs/docs.yml: Added new documentation pages for 'Custom Type Assertions' and renamed 'Multimodal' to 'Multimodal Input'.
  • docs/docs/calling-baml/assertions.mdx: New file detailing custom type assertions and checks, including @assert and @check attributes.
  • docs/docs/calling-baml/dynamic-types.mdx: Updated to include instructions and examples for using dynamic types.
  • docs/docs/calling-baml/multi-modal.mdx: Enhanced with detailed code examples for handling image and audio inputs.
  • docs/docs/snippets/class.mdx: Introduced @@assert and @@check attributes for class-level and field-level validation.

6 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings

docs/docs/calling-baml/assertions.mdx Outdated Show resolved Hide resolved
docs/docs/calling-baml/assertions.mdx Outdated Show resolved Hide resolved
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

Updated the documentation to replace @constraint with @assert for consistency and added examples for @assert and @check attributes.

  • docs/docs/calling-baml/assertions.mdx: Replaced @constraint with @assert and added examples for field-level, block-level, and dynamic input assertions.
  • docs/docs/calling-baml/assertions.mdx: Introduced @check for non-exception raising checks.
  • docs/docs/calling-baml/assertions.mdx: Improved clarity and accuracy of custom type assertions documentation.

1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

Updated documentation to align terminology and enhance clarity on custom type validations and checks.

  • docs/docs.yml: Renamed navigation entry from 'Custom Type Assertions' to 'Custom Type Validations'.
  • docs/docs/calling-baml/assertions.mdx: Expanded explanations on @assert and @check attributes, including examples and usage distinctions.
  • docs/docs/snippets/class.mdx: Added sections for 'Contextual Attributes' and 'Validation Attributes', detailing @@dynamic, @@assert, and @@check attributes with examples.

Ensure all references to 'Custom Type Assertions' are updated to 'Custom Type Validations' for consistency.

3 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

Updated documentation to improve clarity and organization for custom type assertions and checks.

  • docs/docs.yml: Renamed 'Advanced BAML Snippets' to 'Advanced BAML Features' for better categorization.
  • docs/docs/calling-baml/assertions.mdx: Enhanced examples and explanations for @assert and @check attributes, including advanced validation scenarios.
  • docs/docs/snippets/prompt-syntax/what-is-jinja.mdx: Updated references from Jinja to Minijinja and adjusted syntax highlighting to reflect this change.

3 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

This pull request introduces several updates to enhance the parser's handling of identifiers and string literals, along with updates to test cases and model configurations.

  • engine/baml-lib/schema-ast/src/parser/datamodel.pest: Added identifier to expression rule and removed unquoted_string_literal from string_literal rule.
  • engine/baml-lib/schema-ast/src/parser/parse_expression.rs: Added a match arm for Rule::identifier in parse_expression function.
  • integ-tests/baml_src/clients.baml: Enclosed model names in double quotes for consistency.
  • integ-tests/baml_src/main.baml: Updated output_type values for lang_python and lang_ruby generators to string literals.
  • integ-tests/baml_src/test-files/providers/providers.baml: Added PromptTestOpenAI function using GPT4Turbo client and updated TestProvider test case.

7 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The recent changes focus on improving the readability and robustness of test files by updating string formatting conventions.

  • integ-tests/baml_src/test-files/aliases/enums.baml: Enhanced enum descriptions by using multi-line string syntax for better readability.
  • integ-tests/baml_src/test-files/functions/input/named-args/single/named-audio.baml: Enclosed URL in double quotes to ensure proper string interpretation within BAML syntax.

2 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings

@anish-palakurthi anish-palakurthi changed the title Spec docs for custom type assertions and checks Custom Type Validations Jul 24, 2024
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The datamodel.pest file has been significantly updated to unify parsing rules for various constructs, aiming to streamline the process and improve maintainability.

  • File: engine/baml-lib/schema-ast/src/parser/datamodel.pest
    • Unified parsing rules for classes, enums, functions, tests, clients, and generators.
    • Introduced new parsing rules and removed redundant ones.
    • Potential for parsing errors due to the complexity of the grammar; thorough testing is recommended.
    • No immediate security issues detected, but careful review is necessary to ensure intended behavior is maintained.

1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The datamodel.pest file has been updated to enhance the flexibility and readability of the parser by reordering elements and introducing new rules.

  • File: engine/baml-lib/schema-ast/src/parser/datamodel.pest
    • Reordered elements in the schema rule, potentially affecting parsing order.
    • Added identifier to class_enum_keyword, broadening the scope of class/enum keywords.
    • Introduced new rules for openParan and closeParan to replace direct usage of parentheses.
    • Changes aim to improve parser flexibility and readability but require thorough testing to avoid unforeseen consequences.

1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The pull request introduces significant refactoring and removal of modules in the AST representation, focusing on consolidating type identifiers and simplifying the codebase.

  • File: engine/baml-lib/schema-ast/src/ast.rs

    • Removed multiple modules and introduced TypeExpId and ValExpId for streamlined type identifiers.
    • Updated TopId enum to include TestCase and RetryPolicy variants.
  • File: engine/baml-lib/schema-ast/src/ast/attribute.rs

    • Removed several AttributeContainer enum variants related to Variant configurations and serializers.
  • File: engine/baml-lib/schema-ast/src/ast/field.rs

    • Changed field_type in Field struct to Option<FieldType> for optional field types.
  • File: engine/baml-lib/schema-ast/src/parser/parse_named_args_list.rs (added)

    • Introduced functions parse_named_arguement_list and parse_function_arg for parsing named argument lists and function arguments.
  • File: engine/baml-lib/schema-ast/src/parser/parse_type_expression.rs (added)

    • Added parse_type_expression function to handle type expressions in the BAML schema.

31 file(s) reviewed, 5 comment(s)
Edit PR Review Bot Settings


use super::helpers::Pair;

pub fn parse_named_arguement_list(
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling: Typo in function name: 'parse_named_arguement_list' should be 'parse_named_argument_list'

name.span().clone(),
)),
(None, _) => {
unreachable!("parse_function_field_type: unexpected rule:")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Style: The error message in unreachable! should be more descriptive to aid debugging.

Rule::ENUM_KEYWORD => sub_type = Some(SubType::Enum),
Rule::BLOCK_OPEN | Rule::BLOCK_CLOSE => {}
Rule::identifier => name = Some(parse_identifier(current, diagnostics)),
Rule::named_argument_list => match parse_named_arguement_list(current, diagnostics) {
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Spelling: Typo in function name: parse_named_arguement_list should be parse_named_argument_list

Rule::BLOCK_OPEN | Rule::BLOCK_CLOSE => {}
Rule::identifier => name = Some(parse_identifier(current, diagnostics)),
Rule::named_argument_list => match parse_named_arguement_list(current, diagnostics) {
Ok(BlockArgs::Named(arg)) => input = Some(BlockArgs::Named(arg)),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic: Variable input is not defined. This will cause a runtime error.

attributes,
documentation: doc_comment.and_then(parse_comment_block),
span: diagnostics.span(pair_span),
sub_type: None,
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Logic: sub_type is always set to None in the final TypeExpression. Consider setting it to the parsed sub_type value.

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The pull request introduces custom type validations and significant refactoring to simplify the codebase and improve maintainability.

  • File: engine/baml-lib/schema-ast/src/ast.rs

    • Refactored SchemaAst to use ValExpId instead of multiple specific ID types.
    • Updated TopId enum to reflect the new ValExpId type.
  • File: engine/baml-lib/schema-ast/src/ast/attribute.rs

    • Removed From trait implementations and Index trait match arms for VariantConfigId, VariantSerializerId, and SerializerFieldId.
  • File: engine/baml-lib/schema-ast/src/ast/top.rs

    • Consolidated multiple methods into as_type_expression and as_value_exp.
    • Introduced as_template_string method.
  • File: engine/baml-lib/schema-ast/src/parser.rs

    • Removed parse_config module import and added new module imports for parsing.
  • File: engine/baml-lib/schema-ast/src/parser/parse_field.rs

    • Modified field_type attribute in Field struct to Option<FieldType>.

5 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The recent changes introduce custom type validations and improve type handling across various files.

  • File: engine/baml-lib/parser-database/src/attributes/to_string_attribute.rs

    • Reordered description attribute handling for logical flow.
  • File: engine/baml-lib/parser-database/src/types/post_prompt.rs

    • Enhanced validation logic for variable paths in BAML prompt parser.
  • File: engine/baml-lib/parser-database/src/walkers/class.rs

    • Filtered out fields without types in add_to_types method.
  • File: engine/baml-lib/parser-database/src/walkers/field.rs

    • Modified r#type method to return Option<FieldType> and updated serialize_data method accordingly.
  • File: integ-tests/baml_src/test-files/aliases/classes.baml

    • Added new classes and enum with descriptions for fields and enum values.

15 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The recent changes focus on custom type validations and refining error messages across various BAML files and Rust source code.

  • File: engine/baml-lib/baml-core/src/ir/repr.rs

    • Added debug print statement for class attributes during IR transformation.
  • File: engine/baml-lib/baml/tests/validation_files/class/attributes.baml

    • Introduced TestClassAlias with string properties having aliases and descriptions.
  • File: engine/baml-lib/baml/tests/validation_files/class/map_types.baml

    • Updated validation error messages for map key types.
  • File: engine/baml-lib/baml/tests/validation_files/functions_v2/duplicate_names.baml

    • Improved error message specificity for duplicate function definitions.
  • File: integ-tests/python/baml_client/partial_types.py

    • Removed multiple classes, focusing on essential types for streaming purposes.

92 file(s) reviewed, 8 comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The recent changes focus on enhancing custom type validations and refining attribute handling for classes and enums in the BAML schema.

  • File: engine/baml-lib/baml-core/src/ir/repr.rs

    • Added SubType import and debug print statements for EnumWalker and ClassWalker attributes.
  • File: engine/baml-lib/parser-database/src/attributes/mod.rs

    • Refactored resolve_type_exp_block_attributes to include SubType parameter for better attribute resolution.
  • File: engine/baml-lib/parser-database/src/walkers/class.rs

    • Introduced get_default_attributes method in ClassWalker to retrieve default attributes based on SubType.
  • File: engine/baml-lib/parser-database/src/walkers/enum.rs

    • Removed get_default_attributes from EnumWalker and added debug prints in EnumValueWalker.
  • File: engine/baml-lib/parser-database/src/walkers/field.rs

    • Added debug print statements to get_default_attributes method for tracing execution.

5 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings

engine/baml-lib/parser-database/src/walkers/class.rs Outdated Show resolved Hide resolved
engine/baml-lib/parser-database/src/walkers/enum.rs Outdated Show resolved Hide resolved
Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The recent changes introduce custom type validations and refine attribute handling for classes and enums in the BAML schema.

  • File: engine/baml-lib/baml-core/src/ir/json_schema.rs

    • Fixed json_schema method call for (&FunctionNode, bool) to ensure correct JSON schema generation.
  • File: engine/baml-lib/baml-core/src/validate/validation_pipeline/validations/functions.rs

    • Enhanced validation logic for functions, including template strings and argument types.
  • File: engine/baml-lib/baml-core/src/validate/validation_pipeline/validations/types.rs

    • Added validation for FieldType::Primitive and FieldType::Symbol types.
  • File: integ-tests/baml_src/fiddle-examples/chain-of-thought.baml

    • Added GetOrderInfo function using GPT4 for extracting order information with step-by-step reasoning.
  • File: integ-tests/python/baml_client/async_client.py

    • Introduced BamlAsyncClient class with asynchronous methods for interacting with the BAML runtime.

89 file(s) reviewed, 3 comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The recent changes introduce custom type validations and refine parsing logic for fields and type expressions in the BAML schema.

  • File: engine/baml-lib/schema-ast/src/parser/parse_field.rs

    • Added is_enum parameter to parse_expr_as_type for conditional parsing of field_type_chain.
  • File: engine/baml-lib/schema-ast/src/parser/parse_type_expression_block.rs

    • Introduced parse_type_expression_block function to handle type expression parsing with subtypes (Class, Enum, Other).
  • File: integ-tests/baml_src/test-files/aliases/enums.baml

    • Added TestEnum with aliases and descriptions, and FnTestAliasedEnumOutput function for classification using GPT35 client.

3 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The recent changes focus on enhancing logging for validation processes, simplifying client definitions, and removing several test files and functions.

  • File: engine/baml-lib/baml-core/src/validate/validation_pipeline/validations/functions.rs

    • Added detailed logging for validation processes.
  • File: engine/baml-lib/baml/tests/validation_files/functions_v2/tests/parameters

    • Introduced TestFnNamedArgsSingleBool function for boolean argument validation.
  • File: engine/baml-lib/jinja/src/evaluate_type/expr.rs

    • Added logging to tracker_visit_expr for better debugging.
  • File: integ-tests/baml_src/clients.baml

    • Simplified to a single client definition for Vertex.
  • File: Multiple files in integ-tests/baml_src/fiddle-examples and test-files directories

    • Removed numerous test files and functions, impacting various validation and client configurations. Ensure no dependencies are broken.

59 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The recent changes introduce custom type validations, new functions, and enhanced logging for validation processes.

  • File: engine/baml-lib/baml-core/src/validate/validation_pipeline/validations/functions.rs

    • Removed logging statements to reduce verbosity and improve performance.
  • File: integ-tests/baml_src/clients.baml

    • Added multiple LLM clients and retry policies for integration tests.
  • File: integ-tests/baml_src/fiddle-examples/chain-of-thought.baml

    • Defined GetOrderInfo function for extracting order details from emails using GPT4.
  • File: integ-tests/baml_src/fiddle-examples/classify-message.baml

    • Introduced ClassifyMessage function for classifying input strings into predefined categories using GPT-4.
  • File: integ-tests/baml_src/test-files/functions/output/class-dynamic.baml

    • Added dynamic types and functions for runtime modifications and data extraction using GPT4.

55 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings

Copy link

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

PR Summary

(updates since last review)

The recent changes focus on refining type validations, enhancing logging, and improving type handling across various files.

  • File: engine/baml-lib/baml-core/src/validate/validation_pipeline/validations/functions.rs

    • Improved type validation for function arguments and template strings.
    • Enhanced error handling for template validation.
  • File: engine/baml-lib/parser-database/src/walkers/mod.rs

    • Enhanced to_jinja_type function to handle FieldType::Symbol more accurately.
  • File: integ-tests/baml_src/test-files/functions/input/named-args/syntax.baml

    • Added a new test file for named arguments syntax in functions.
  • File: integ-tests/python/baml_client/type_builder.py

    • Added dynamic class and enum builders for runtime type creation.
  • File: integ-tests/ruby/baml_client/type-registry.rb

    • Introduced a TypeBuilder class for managing types within the BAML system.

24 file(s) reviewed, 5 comment(s)
Edit PR Review Bot Settings

};

use crate::{assert_correct_parser, ast::*};
use crate::{ast::TypeExpressionBlock, parser::parse_field::parse_expr_as_type}; // Add this line to import DatamodelParser
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Ensure parse_expr_as_type is thoroughly tested for integration.

"client" | "client<llm>" => sub_type = Some(ValueExprBlockType::Client),
"retry_policy" => sub_type = Some(ValueExprBlockType::RetryPolicy),
"generator" => sub_type = Some(ValueExprBlockType::Generator),
_ => panic!("Unexpected value expression keyword: {}", current.as_str()),
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Using panic! for unexpected keywords can terminate the program unexpectedly. Consider using a more graceful error handling approach.

fields,
documentation: doc_comment.and_then(parse_comment_block),
span: diagnostics.span(pair_span),
block_type: sub_type.unwrap_or(ValueExprBlockType::Function), // Unwrap or provide a default
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

logic: Unwrapping sub_type without a default value can cause a panic if sub_type is None. Ensure that sub_type is always set or provide a default value.

Comment on lines +65 to +67
if varargs.any?

raise ArgumentError.new("AaaSamOutputFormat may only be called with keyword arguments")
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Consider providing more specific error messages for better debugging.

Comment on lines +249 to 250
baml_options: T::Hash[Symbol, T.any(Baml::TypeBuilder, Baml::ClientRegistry)]
).returns(String)
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

style: Consider splitting complex methods into smaller, more manageable functions.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant