Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Makes IonSchemaElement use borrowed Elements #222

Merged
merged 2 commits into from
Oct 15, 2024
Merged

Makes IonSchemaElement use borrowed Elements #222

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
2 commits
Select commit Hold shift + click to select a range
ec65db9
Makes IonSchemaElement borrow rather than own
popematt Jul 9, 2024
e0f485b
cargo fmt fixes
popematt Oct 14, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion ion-schema-tests-runner/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@ proc-macro = true
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
ion-rs = "1.0.0-rc.7"
ion-rs = "1.0.0-rc.8"
ion-schema = { path = "../ion-schema" }
quote = "1.0.21"
syn = "1.0.102"
Expand Down
17 changes: 7 additions & 10 deletions ion-schema-tests-runner/src/generator.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -321,18 +321,15 @@ fn generate_preamble(root_dir_path: &Path) -> TokenStream {

/// Asserts that a value is or is not valid for a given ISL type.
fn __assert_value_validity_for_type(value_ion: &str, schema_id: &str, type_id: &str, expect_valid: bool) -> Result<(), String> {
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do you happen to know why we're using a __ prefix rather than, say, making a new submodule to house these functions?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wanted to make sure that we didn't have any name conflicts with a generated test case, so I added a __ prefix. I could have added a new submodule, but that probably also would have needed something to reduce the likelihood of a name conflict with a generated test function.

let schema = __new_schema_system().load_schema(schema_id).unwrap();
let isl_type = schema.get_type(type_id).unwrap();
let value: ion_rs::Element = ion_rs::Element::read_one(value_ion.as_bytes()).unwrap();
let schema = __new_schema_system().load_schema(schema_id).expect(&format!("Expected to load schema: {}", schema_id));
let isl_type = schema.get_type(type_id).expect(&format!("Expected to get type: {}", type_id));
let value: ion_rs::Element = ion_rs::Element::read_one(value_ion.as_bytes()).expect(&format!("Expected to be able to read value: {}", value_ion));
Comment on lines +324 to +326
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗺️ Drive-by improvement to get better failure messages in the test runner.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not very familiar with this part of the code--it looks like this is a method that only gets run in testing, so performance may not matter. If so, you can disregard.

These expect(&format!(...)) calls will allocate/format a String even if the previous method calls succeed. We should use something like Result::unwrap_or_else(|| panic!("Expected ...")) to sidestep the cost on the happy path.

https://doc.rust-lang.org/std/result/enum.Result.html#method.unwrap_or_else

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

It's only in testing.

let prepared_value: ion_schema::IonSchemaElement = if value.annotations().contains("document") && value.ion_type() == ion_rs::IonType::SExp {
let element_vec = value.as_sequence()
.unwrap_or_else(|| unreachable!("We already confirmed that this is a s-expression."))
.elements()
.map(|it| it.to_owned())
.collect::<Vec<_>>();
ion_schema::IonSchemaElement::Document(element_vec)
let values = value.as_sequence()
.expect("We already confirmed that this is a s-expression.");
ion_schema::IonSchemaElement::from(ion_schema::AsDocumentHint::as_document(values))
} else {
ion_schema::IonSchemaElement::SingleElement(value)
ion_schema::IonSchemaElement::from(&value)
};
let validation_result = isl_type.validate(prepared_value);
if validation_result.is_ok() == expect_valid {
Expand Down
6 changes: 3 additions & 3 deletions ion-schema-tests-runner/tests/ion-schema-tests-2-0.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -2,11 +2,11 @@ use ion_schema_tests_runner::ion_schema_tests;

ion_schema_tests!(
root = "ion-schema-tests/ion_schema_2_0/",
// Support for ISL 2.0 is not completely implemented yet, so some tests are ignored.
ignored(
// Not fully implemented yet.
"imports",
"constraints::ordered_elements",
"constraints::precision",
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🗺️ It turns out that we don't need to skip the tests for precision any more. However, the fix was in some previous change. All of the other changes in this file are just trying to improve the specificity of the comments.

"constraints::regex::value_should_be_invalid_for_type_regex_unescaped_newline__2_", // https://github.com/amazon-ion/ion-rust/issues/399
// Failing because of https://github.com/amazon-ion/ion-rust/issues/399
"constraints::regex::value_should_be_invalid_for_type_regex_unescaped_newline__2_",
)
);
2 changes: 1 addition & 1 deletion ion-schema/Cargo.toml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -22,7 +22,7 @@ edition = "2021"
# See more keys and their definitions at https://doc.rust-lang.org/cargo/reference/manifest.html

[dependencies]
ion-rs = { version = "1.0.0-rc.7", features = ["experimental-reader-writer"] }
ion-rs = { version = "1.0.0-rc.8", features = ["experimental-reader-writer"] }
thiserror = "1.0"
num-bigint = "0.3"
num-traits = "0.2"
Expand Down
207 changes: 80 additions & 127 deletions ion-schema/src/constraint.rs
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -899,27 +899,24 @@
) -> ValidationResult {
let violations: Vec<Violation> = vec![];

let values: Vec<Element> = match &value {
IonSchemaElement::SingleElement(element) => match element.as_sequence() {
None => {
return Err(Violation::with_violations(
"ordered_elements",
ViolationCode::TypeMismatched,
format!(
"expected list/sexp ion found {}",
if element.is_null() {
format!("{element}")
} else {
format!("{}", element.ion_type())
}
),
ion_path,
violations,
));
}
Some(sequence) => sequence.elements().map(|a| a.to_owned()).collect(),
},
IonSchemaElement::Document(document) => document.to_owned(),
let mut element_iter = match value.as_sequence_iter() {
Some(iter) => iter.peekable(),
None => {
return Err(Violation::with_violations(
"ordered_elements",
ViolationCode::TypeMismatched,
format!(
"expected list, sexp, or document; found {}",
if value.is_null() {
format!("{value}")
} else {
format!("{}", value.ion_schema_type())
}
),
ion_path,
violations,
));
}
};

// build nfa for validation
Expand All @@ -928,7 +925,7 @@
type_store,
);

if !values.is_empty() && nfa_evaluation.nfa.get_final_states().is_empty() {
if element_iter.peek().is_some() && nfa_evaluation.nfa.get_final_states().is_empty() {
return Err(Violation::with_violations(
"ordered_elements",
ViolationCode::TypeMismatched,
Expand All @@ -939,7 +936,7 @@
}

// use nfa_evaluation for validation
nfa_evaluation.validate_ordered_elements(&values, type_store);
nfa_evaluation.validate_ordered_elements(element_iter, type_store);

if !nfa_evaluation.has_final_state(type_store) {
return Err(Violation::with_violations(
Expand Down Expand Up @@ -1173,36 +1170,24 @@
type_store: &TypeStore,
ion_path: &mut IonPath,
) -> ValidationResult {
// Create a peekable iterator for given sequence
let values: Vec<Element> = match &value {
IonSchemaElement::SingleElement(element) => {
match element.value() {
Value::List(ion_sequence) | Value::SExp(ion_sequence) => {
ion_sequence.elements().map(|a| a.to_owned()).collect()
}
Value::Struct(ion_struct) => ion_struct
.fields()
.map(|(name, value)| value.to_owned())
.collect(),
_ => {
// return Violation if value is not an Ion sequence
return Err(Violation::new(
"contains",
ViolationCode::TypeMismatched,
format!(
"expected list/sexp/struct/document found {}",
if element.is_null() {
format!("{element}")
} else {
format!("{}", element.ion_type())
}
),
ion_path,
));
let values: Vec<IonData<&Element>> = if let Some(element_iter) = value.as_sequence_iter() {
element_iter.map(IonData::from).collect()
} else if let Some(strukt) = value.as_struct() {
strukt.fields().map(|(k, v)| v).map(IonData::from).collect()
} else {
return Err(Violation::new(
"contains",
ViolationCode::TypeMismatched,
format!(
"expected list/sexp/struct/document found {}",
if value.is_null() {
format!("{value}")
} else {
format!("{}", value.ion_schema_type())
}
}
}
IonSchemaElement::Document(document) => document.to_owned(),
),
ion_path,
));
};

// add all the missing values found during validation
Expand Down Expand Up @@ -1256,36 +1241,24 @@
ion_path: &mut IonPath,
) -> ValidationResult {
// get the size of given value container
let size = match value {
IonSchemaElement::SingleElement(element) => {
// Check for null container
if element.is_null() {
return Err(Violation::new(
"container_length",
ViolationCode::TypeMismatched,
format!("expected a container found {element}"),
ion_path,
));
}

match element.ion_type() {
IonType::List | IonType::SExp => element.as_sequence().unwrap().len(),
IonType::Struct => element.as_struct().unwrap().iter().count(),
_ => {
// return Violation if value is not an Ion container
return Err(Violation::new(
"container_length",
ViolationCode::TypeMismatched,
format!(
"expected a container (a list/sexp/struct) but found {}",
element.ion_type()
),
ion_path,
));
}
}
}
IonSchemaElement::Document(document) => document.len(),
let size = if let Some(element_iter) = value.as_sequence_iter() {
element_iter.count()
} else if let Some(strukt) = value.as_struct() {
strukt.fields().map(|(k, v)| v).count()

Check warning on line 1247 in ion-schema/src/constraint.rs

View workflow job for this annotation

GitHub Actions / build (ubuntu-latest) 

this call to `map()` won't have an effect on the call to `count()`
} else {
return Err(Violation::new(
"container_length",
ViolationCode::TypeMismatched,
if value.is_null() {
format!("expected a container found {value}")
} else {
format!(
"expected a container (a list/sexp/struct) but found {}",
value.ion_schema_type()
)
},
ion_path,
));
};

// get isl length as a range
Expand Down Expand Up @@ -1438,58 +1411,38 @@
let mut element_set = vec![];

// get elements for given container in the form (ion_path_element, element_value)
let elements: Vec<(IonPathElement, &Element)> = match value {
IonSchemaElement::SingleElement(element) => {
let elements: Vec<(IonPathElement, &Element)> =
if let Some(element_iter) = value.as_sequence_iter() {
element_iter
.enumerate()
.map(|(index, val)| (IonPathElement::Index(index), val))
.collect()
} else if let Some(strukt) = value.as_struct() {
strukt
.fields()
.map(|(name, val)| (IonPathElement::Field(name.to_string()), val))
.collect()
} else {
// Check for null container
if element.is_null() {
if value.is_null() {
return Err(Violation::new(
"element",
ViolationCode::TypeMismatched,
format!("expected a container but found {element}"),
format!("expected a container but found {value}"),
ion_path,
));
}

// validate each element of the given value container
match element.ion_type() {
IonType::List | IonType::SExp => element
.as_sequence()
.unwrap()
.elements()
.enumerate()
.map(|(index, val)| (IonPathElement::Index(index), val))
.collect(),
IonType::Struct => element
.as_struct()
.unwrap()
.iter()
.map(|(field_name, val)| {
(
IonPathElement::Field(field_name.text().unwrap().to_owned()),
val,
)
})
.collect(),
_ => {
// return Violation if value is not an Ion container
return Err(Violation::new(
"element",
ViolationCode::TypeMismatched,
format!(
"expected a container (a list/sexp/struct) but found {}",
element.ion_type()
),
ion_path,
));
}
}
}
IonSchemaElement::Document(document) => document
.iter()
.enumerate()
.map(|(index, val)| (IonPathElement::Index(index), val))
.collect(),
};
// return Violation if value is not an Ion container
return Err(Violation::new(
"element",
ViolationCode::TypeMismatched,
format!(
"expected a container (a list/sexp/struct) but found {}",
value.ion_schema_type()
),
ion_path,
));
};

// validate element constraint
for (ion_path_element, val) in elements {
Expand Down
Loading
Loading