Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add validation for schemas and examples #6

Open
peterbrightwell opened this issue Jul 7, 2020 · 1 comment
Open

Add validation for schemas and examples #6

peterbrightwell opened this issue Jul 7, 2020 · 1 comment

Comments

@peterbrightwell
Copy link
Contributor

Follow-up to AMWA-TV/is-template#3 and AMWA-TV/nmos-render-scripts#1. Text below is from the latter:

In addition to the basic JSON linting performed as part of the CI, it would great to:

  • validate each example file in the examples/ directory in each spec repo against the appropriate schema in the APIs/schemas/ directory
  • validate each schema in the APIs/schemas/ directory against the meta schema (http://json-schema.org/draft-04/schema)

This would have prevented several mistakes previously made during schema design and spec version ups. Seems like there are several reasonable looking JSON Schema engines that could be used during CI.

Tricky bit is identifying which schema should be used to validate which example file, because our JSON formats do not contain embedded metadata to indicate that (and indeed that's recommended practice - instead, out-of-band mechanisms like Link response headers or IANA media-type parameters are usually needed).

We could rely on the RAML type indication but we'd have to make sure all files in examples/ were referenced from the RAML (or at least only validate the ones that were). Unused examples may be a mistake anyway...

The other obvious approach would rely on a .schemas.json file in each repo that simply mapped instances in examples/ to schemas in APIs/schemas, and error if the filenames didn't match up.

@peterbrightwell
Copy link
Contributor Author

peterbrightwell commented Jul 7, 2020

I've made a start (on validate branch... at the moment this is just so I don't lose my experiments) but will need to come back to it at a later time.

validate.sh will look for examples in each RAML file and find their corresponding schemas. Some are included directly, others via a type definition (and unfortunately each spec seems to have done that differently).

None of the validation tools I've tried will correctly handle $refs to local schemas. I propose to use the python jsonschema, and temporarily patch the references with file: prefixes, and any pathname change required.

See also #5

@peterbrightwell peterbrightwell changed the title Add validation Add validation for schemas and examples Jun 2, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant