-
Notifications
You must be signed in to change notification settings - Fork 68
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Custom Type Validations #822
Conversation
anish-palakurthi
commented
Jul 22, 2024
•
edited
Loading
edited
- lookahead spec docs for custom type validations
- significant rework of BAML language server, including changes to the grammar, parser, and IR validation in order to support auto-formatting and type validations.
The latest updates on your projects. Learn more about Vercel for Git ↗︎
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
The pull request introduces new documentation for custom type assertions and checks, along with updates to existing documentation for dynamic types and multimodal input.
- docs/docs.yml: Added new documentation pages for 'Custom Type Assertions' and renamed 'Multimodal' to 'Multimodal Input'.
- docs/docs/calling-baml/assertions.mdx: New file detailing custom type assertions and checks, including
@assert
and@check
attributes. - docs/docs/calling-baml/dynamic-types.mdx: Updated to include instructions and examples for using dynamic types.
- docs/docs/calling-baml/multi-modal.mdx: Enhanced with detailed code examples for handling image and audio inputs.
- docs/docs/snippets/class.mdx: Introduced
@@assert
and@@check
attributes for class-level and field-level validation.
6 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
Updated the documentation to replace @constraint
with @assert
for consistency and added examples for @assert
and @check
attributes.
- docs/docs/calling-baml/assertions.mdx: Replaced
@constraint
with@assert
and added examples for field-level, block-level, and dynamic input assertions. - docs/docs/calling-baml/assertions.mdx: Introduced
@check
for non-exception raising checks. - docs/docs/calling-baml/assertions.mdx: Improved clarity and accuracy of custom type assertions documentation.
1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
Updated documentation to align terminology and enhance clarity on custom type validations and checks.
- docs/docs.yml: Renamed navigation entry from 'Custom Type Assertions' to 'Custom Type Validations'.
- docs/docs/calling-baml/assertions.mdx: Expanded explanations on
@assert
and@check
attributes, including examples and usage distinctions. - docs/docs/snippets/class.mdx: Added sections for 'Contextual Attributes' and 'Validation Attributes', detailing
@@dynamic
,@@assert
, and@@check
attributes with examples.
Ensure all references to 'Custom Type Assertions' are updated to 'Custom Type Validations' for consistency.
3 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
Updated documentation to improve clarity and organization for custom type assertions and checks.
- docs/docs.yml: Renamed 'Advanced BAML Snippets' to 'Advanced BAML Features' for better categorization.
- docs/docs/calling-baml/assertions.mdx: Enhanced examples and explanations for
@assert
and@check
attributes, including advanced validation scenarios. - docs/docs/snippets/prompt-syntax/what-is-jinja.mdx: Updated references from Jinja to Minijinja and adjusted syntax highlighting to reflect this change.
3 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
This pull request introduces several updates to enhance the parser's handling of identifiers and string literals, along with updates to test cases and model configurations.
- engine/baml-lib/schema-ast/src/parser/datamodel.pest: Added
identifier
toexpression
rule and removedunquoted_string_literal
fromstring_literal
rule. - engine/baml-lib/schema-ast/src/parser/parse_expression.rs: Added a match arm for
Rule::identifier
inparse_expression
function. - integ-tests/baml_src/clients.baml: Enclosed model names in double quotes for consistency.
- integ-tests/baml_src/main.baml: Updated
output_type
values forlang_python
andlang_ruby
generators to string literals. - integ-tests/baml_src/test-files/providers/providers.baml: Added
PromptTestOpenAI
function usingGPT4Turbo
client and updatedTestProvider
test case.
7 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The recent changes focus on improving the readability and robustness of test files by updating string formatting conventions.
- integ-tests/baml_src/test-files/aliases/enums.baml: Enhanced enum descriptions by using multi-line string syntax for better readability.
- integ-tests/baml_src/test-files/functions/input/named-args/single/named-audio.baml: Enclosed URL in double quotes to ensure proper string interpretation within BAML syntax.
2 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The datamodel.pest
file has been significantly updated to unify parsing rules for various constructs, aiming to streamline the process and improve maintainability.
- File:
engine/baml-lib/schema-ast/src/parser/datamodel.pest
- Unified parsing rules for classes, enums, functions, tests, clients, and generators.
- Introduced new parsing rules and removed redundant ones.
- Potential for parsing errors due to the complexity of the grammar; thorough testing is recommended.
- No immediate security issues detected, but careful review is necessary to ensure intended behavior is maintained.
1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The datamodel.pest
file has been updated to enhance the flexibility and readability of the parser by reordering elements and introducing new rules.
- File:
engine/baml-lib/schema-ast/src/parser/datamodel.pest
- Reordered elements in the
schema
rule, potentially affecting parsing order. - Added
identifier
toclass_enum_keyword
, broadening the scope of class/enum keywords. - Introduced new rules for
openParan
andcloseParan
to replace direct usage of parentheses. - Changes aim to improve parser flexibility and readability but require thorough testing to avoid unforeseen consequences.
- Reordered elements in the
1 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The pull request introduces significant refactoring and removal of modules in the AST representation, focusing on consolidating type identifiers and simplifying the codebase.
-
File:
engine/baml-lib/schema-ast/src/ast.rs
- Removed multiple modules and introduced
TypeExpId
andValExpId
for streamlined type identifiers. - Updated
TopId
enum to includeTestCase
andRetryPolicy
variants.
- Removed multiple modules and introduced
-
File:
engine/baml-lib/schema-ast/src/ast/attribute.rs
- Removed several
AttributeContainer
enum variants related toVariant
configurations and serializers.
- Removed several
-
File:
engine/baml-lib/schema-ast/src/ast/field.rs
- Changed
field_type
inField
struct toOption<FieldType>
for optional field types.
- Changed
-
File:
engine/baml-lib/schema-ast/src/parser/parse_named_args_list.rs
(added)- Introduced functions
parse_named_arguement_list
andparse_function_arg
for parsing named argument lists and function arguments.
- Introduced functions
-
File:
engine/baml-lib/schema-ast/src/parser/parse_type_expression.rs
(added)- Added
parse_type_expression
function to handle type expressions in the BAML schema.
- Added
31 file(s) reviewed, 5 comment(s)
Edit PR Review Bot Settings
|
||
use super::helpers::Pair; | ||
|
||
pub fn parse_named_arguement_list( |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Spelling: Typo in function name: 'parse_named_arguement_list' should be 'parse_named_argument_list'
name.span().clone(), | ||
)), | ||
(None, _) => { | ||
unreachable!("parse_function_field_type: unexpected rule:") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Style: The error message in unreachable!
should be more descriptive to aid debugging.
Rule::ENUM_KEYWORD => sub_type = Some(SubType::Enum), | ||
Rule::BLOCK_OPEN | Rule::BLOCK_CLOSE => {} | ||
Rule::identifier => name = Some(parse_identifier(current, diagnostics)), | ||
Rule::named_argument_list => match parse_named_arguement_list(current, diagnostics) { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Spelling: Typo in function name: parse_named_arguement_list
should be parse_named_argument_list
Rule::BLOCK_OPEN | Rule::BLOCK_CLOSE => {} | ||
Rule::identifier => name = Some(parse_identifier(current, diagnostics)), | ||
Rule::named_argument_list => match parse_named_arguement_list(current, diagnostics) { | ||
Ok(BlockArgs::Named(arg)) => input = Some(BlockArgs::Named(arg)), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logic: Variable input
is not defined. This will cause a runtime error.
attributes, | ||
documentation: doc_comment.and_then(parse_comment_block), | ||
span: diagnostics.span(pair_span), | ||
sub_type: None, |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Logic: sub_type
is always set to None
in the final TypeExpression
. Consider setting it to the parsed sub_type
value.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The pull request introduces custom type validations and significant refactoring to simplify the codebase and improve maintainability.
-
File:
engine/baml-lib/schema-ast/src/ast.rs
- Refactored
SchemaAst
to useValExpId
instead of multiple specific ID types. - Updated
TopId
enum to reflect the newValExpId
type.
- Refactored
-
File:
engine/baml-lib/schema-ast/src/ast/attribute.rs
- Removed
From
trait implementations andIndex
trait match arms forVariantConfigId
,VariantSerializerId
, andSerializerFieldId
.
- Removed
-
File:
engine/baml-lib/schema-ast/src/ast/top.rs
- Consolidated multiple methods into
as_type_expression
andas_value_exp
. - Introduced
as_template_string
method.
- Consolidated multiple methods into
-
File:
engine/baml-lib/schema-ast/src/parser.rs
- Removed
parse_config
module import and added new module imports for parsing.
- Removed
-
File:
engine/baml-lib/schema-ast/src/parser/parse_field.rs
- Modified
field_type
attribute inField
struct toOption<FieldType>
.
- Modified
5 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The recent changes introduce custom type validations and improve type handling across various files.
-
File:
engine/baml-lib/parser-database/src/attributes/to_string_attribute.rs
- Reordered
description
attribute handling for logical flow.
- Reordered
-
File:
engine/baml-lib/parser-database/src/types/post_prompt.rs
- Enhanced validation logic for variable paths in BAML prompt parser.
-
File:
engine/baml-lib/parser-database/src/walkers/class.rs
- Filtered out fields without types in
add_to_types
method.
- Filtered out fields without types in
-
File:
engine/baml-lib/parser-database/src/walkers/field.rs
- Modified
r#type
method to returnOption<FieldType>
and updatedserialize_data
method accordingly.
- Modified
-
File:
integ-tests/baml_src/test-files/aliases/classes.baml
- Added new classes and enum with descriptions for fields and enum values.
15 file(s) reviewed, no comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The recent changes focus on custom type validations and refining error messages across various BAML files and Rust source code.
-
File:
engine/baml-lib/baml-core/src/ir/repr.rs
- Added debug print statement for class attributes during IR transformation.
-
File:
engine/baml-lib/baml/tests/validation_files/class/attributes.baml
- Introduced
TestClassAlias
with string properties having aliases and descriptions.
- Introduced
-
File:
engine/baml-lib/baml/tests/validation_files/class/map_types.baml
- Updated validation error messages for map key types.
-
File:
engine/baml-lib/baml/tests/validation_files/functions_v2/duplicate_names.baml
- Improved error message specificity for duplicate function definitions.
-
File:
integ-tests/python/baml_client/partial_types.py
- Removed multiple classes, focusing on essential types for streaming purposes.
92 file(s) reviewed, 8 comment(s)
Edit PR Review Bot Settings
engine/baml-lib/schema-ast/src/parser/parse_type_expression_block.rs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The recent changes focus on enhancing custom type validations and refining attribute handling for classes and enums in the BAML schema.
-
File:
engine/baml-lib/baml-core/src/ir/repr.rs
- Added
SubType
import and debug print statements forEnumWalker
andClassWalker
attributes.
- Added
-
File:
engine/baml-lib/parser-database/src/attributes/mod.rs
- Refactored
resolve_type_exp_block_attributes
to includeSubType
parameter for better attribute resolution.
- Refactored
-
File:
engine/baml-lib/parser-database/src/walkers/class.rs
- Introduced
get_default_attributes
method inClassWalker
to retrieve default attributes based onSubType
.
- Introduced
-
File:
engine/baml-lib/parser-database/src/walkers/enum.rs
- Removed
get_default_attributes
fromEnumWalker
and added debug prints inEnumValueWalker
.
- Removed
-
File:
engine/baml-lib/parser-database/src/walkers/field.rs
- Added debug print statements to
get_default_attributes
method for tracing execution.
- Added debug print statements to
5 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The recent changes introduce custom type validations and refine attribute handling for classes and enums in the BAML schema.
-
File:
engine/baml-lib/baml-core/src/ir/json_schema.rs
- Fixed
json_schema
method call for(&FunctionNode, bool)
to ensure correct JSON schema generation.
- Fixed
-
File:
engine/baml-lib/baml-core/src/validate/validation_pipeline/validations/functions.rs
- Enhanced validation logic for functions, including template strings and argument types.
-
File:
engine/baml-lib/baml-core/src/validate/validation_pipeline/validations/types.rs
- Added validation for
FieldType::Primitive
andFieldType::Symbol
types.
- Added validation for
-
File:
integ-tests/baml_src/fiddle-examples/chain-of-thought.baml
- Added
GetOrderInfo
function using GPT4 for extracting order information with step-by-step reasoning.
- Added
-
File:
integ-tests/python/baml_client/async_client.py
- Introduced
BamlAsyncClient
class with asynchronous methods for interacting with the BAML runtime.
- Introduced
89 file(s) reviewed, 3 comment(s)
Edit PR Review Bot Settings
integ-tests/baml_src/test-files/functions/input/named-args/single/named-image-list.baml
Show resolved
Hide resolved
integ-tests/baml_src/test-files/functions/input/named-args/single/named-map-string-to-map.baml
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The recent changes introduce custom type validations and refine parsing logic for fields and type expressions in the BAML schema.
-
File:
engine/baml-lib/schema-ast/src/parser/parse_field.rs
- Added
is_enum
parameter toparse_expr_as_type
for conditional parsing offield_type_chain
.
- Added
-
File:
engine/baml-lib/schema-ast/src/parser/parse_type_expression_block.rs
- Introduced
parse_type_expression_block
function to handle type expression parsing with subtypes (Class
,Enum
,Other
).
- Introduced
-
File:
integ-tests/baml_src/test-files/aliases/enums.baml
- Added
TestEnum
with aliases and descriptions, andFnTestAliasedEnumOutput
function for classification usingGPT35
client.
- Added
3 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings
engine/baml-lib/schema-ast/src/parser/parse_type_expression_block.rs
Outdated
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The recent changes focus on enhancing logging for validation processes, simplifying client definitions, and removing several test files and functions.
-
File:
engine/baml-lib/baml-core/src/validate/validation_pipeline/validations/functions.rs
- Added detailed logging for validation processes.
-
File:
engine/baml-lib/baml/tests/validation_files/functions_v2/tests/parameters
- Introduced
TestFnNamedArgsSingleBool
function for boolean argument validation.
- Introduced
-
File:
engine/baml-lib/jinja/src/evaluate_type/expr.rs
- Added logging to
tracker_visit_expr
for better debugging.
- Added logging to
-
File:
integ-tests/baml_src/clients.baml
- Simplified to a single client definition for
Vertex
.
- Simplified to a single client definition for
-
File: Multiple files in
integ-tests/baml_src/fiddle-examples
andtest-files
directories- Removed numerous test files and functions, impacting various validation and client configurations. Ensure no dependencies are broken.
59 file(s) reviewed, 1 comment(s)
Edit PR Review Bot Settings
engine/baml-lib/baml/tests/validation_files/functions_v2/tests/parameters
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The recent changes introduce custom type validations, new functions, and enhanced logging for validation processes.
-
File:
engine/baml-lib/baml-core/src/validate/validation_pipeline/validations/functions.rs
- Removed logging statements to reduce verbosity and improve performance.
-
File:
integ-tests/baml_src/clients.baml
- Added multiple LLM clients and retry policies for integration tests.
-
File:
integ-tests/baml_src/fiddle-examples/chain-of-thought.baml
- Defined
GetOrderInfo
function for extracting order details from emails using GPT4.
- Defined
-
File:
integ-tests/baml_src/fiddle-examples/classify-message.baml
- Introduced
ClassifyMessage
function for classifying input strings into predefined categories using GPT-4.
- Introduced
-
File:
integ-tests/baml_src/test-files/functions/output/class-dynamic.baml
- Added dynamic types and functions for runtime modifications and data extraction using GPT4.
55 file(s) reviewed, 2 comment(s)
Edit PR Review Bot Settings
integ-tests/baml_src/test-files/functions/input/named-args/single/named-image-list.baml
Show resolved
Hide resolved
integ-tests/baml_src/test-files/functions/input/named-args/single/named-map-string-to-map.baml
Show resolved
Hide resolved
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
PR Summary
(updates since last review)
The recent changes focus on refining type validations, enhancing logging, and improving type handling across various files.
-
File:
engine/baml-lib/baml-core/src/validate/validation_pipeline/validations/functions.rs
- Improved type validation for function arguments and template strings.
- Enhanced error handling for template validation.
-
File:
engine/baml-lib/parser-database/src/walkers/mod.rs
- Enhanced
to_jinja_type
function to handleFieldType::Symbol
more accurately.
- Enhanced
-
File:
integ-tests/baml_src/test-files/functions/input/named-args/syntax.baml
- Added a new test file for named arguments syntax in functions.
-
File:
integ-tests/python/baml_client/type_builder.py
- Added dynamic class and enum builders for runtime type creation.
-
File:
integ-tests/ruby/baml_client/type-registry.rb
- Introduced a
TypeBuilder
class for managing types within the BAML system.
- Introduced a
24 file(s) reviewed, 5 comment(s)
Edit PR Review Bot Settings
}; | ||
|
||
use crate::{assert_correct_parser, ast::*}; | ||
use crate::{ast::TypeExpressionBlock, parser::parse_field::parse_expr_as_type}; // Add this line to import DatamodelParser |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: Ensure parse_expr_as_type
is thoroughly tested for integration.
"client" | "client<llm>" => sub_type = Some(ValueExprBlockType::Client), | ||
"retry_policy" => sub_type = Some(ValueExprBlockType::RetryPolicy), | ||
"generator" => sub_type = Some(ValueExprBlockType::Generator), | ||
_ => panic!("Unexpected value expression keyword: {}", current.as_str()), |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: Using panic!
for unexpected keywords can terminate the program unexpectedly. Consider using a more graceful error handling approach.
fields, | ||
documentation: doc_comment.and_then(parse_comment_block), | ||
span: diagnostics.span(pair_span), | ||
block_type: sub_type.unwrap_or(ValueExprBlockType::Function), // Unwrap or provide a default |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
logic: Unwrapping sub_type
without a default value can cause a panic if sub_type
is None
. Ensure that sub_type
is always set or provide a default value.
if varargs.any? | ||
|
||
raise ArgumentError.new("AaaSamOutputFormat may only be called with keyword arguments") |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: Consider providing more specific error messages for better debugging.
baml_options: T::Hash[Symbol, T.any(Baml::TypeBuilder, Baml::ClientRegistry)] | ||
).returns(String) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
style: Consider splitting complex methods into smaller, more manageable functions.