-
Notifications
You must be signed in to change notification settings - Fork 46
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
$schema should point to meta-schema incorporating our changes to JSON Schema #566
Comments
It may be possible to simply re-use this (reformatting that info as JSON): https://github.com/open-contracting/standard-maintenance-scripts/blob/master/tests/test_json.py#L42-L57 |
@jpmckinney thanks for the link that was helpful. In pull request #611 in the schema directory there is an adaptation of the draft-04 schema which is more strict (uses additionalProperties: false) and covers our use. It is also configured to run in the tests so travis will pick up issues. There is still some work todo on adding title and description tags to the fields we have added which @timgdavies said he would look at. Also, $ref and format properties had to be added to make the additionalProperties: false work. I nonetheless have concerns about using this meta-schema in the the $schema tag. Also it could be considered a breaking change and "could" break some libraries as they would no longer have a link to an official draft schema to validate against. As our schemas are also valid against draft-4 then I think we should keep it in our schemas. draft-4 meta-schema itself does not validate itself (if you add additionalProperties: false due to the format and $ref) so this must be expected usage. |
Hmm, but isn't the JSON document we are extending (http://json-schema.org/draft-04/schema#) the 'official draft schema to validate against'? Our meta-schema, itself, would have a Are libraries just looking at the string 'http://json-schema.org/draft-04/schema#' and deciding on a bunch of rules to apply, independent of the contents of the JSON document? I know that JSON Schema has non-schema rules about how 'format' is validated, and about what words like 'maxItems' mean, but I don't think those rules and semantics have changed across versions of JSON Schema. In other words, libraries shouldn't need to have different code paths for different versions. I contributed to a Ruby JSON Schema validator, and it didn't have special logic that followed from which schema version was being used. http://json-schema.org/latest/json-schema-core.html#rfc.section.6.4 says:
|
From my understanding there are non-schema rules across drafts that do change and that we can not be assured of them not in future. However, you are correct that libraries could look at the meta-schemas $schema to determine the official version to look for but I am not sure they would do that. On reflection, I do not mind if we do this, but we should at least wait for minor bump in OCDS version number if we do. There is quite a bit of work around adding the meta-schema.json to the official schema locations that has to be worked out and making sure the references in the id/&schema tags are in sync. |
I mainly just want to kick the tires on this, so that we don't do things one way, and then decide at a later date that we should have done it the way the JSON Schema draft recommends all along. Could you describe the specific work that would have to be done in a minor bump, while it's still fresh? |
In a minor version bump we should do the following: Governance
Documentation
Technical Changes
|
Thanks! I think JSON Schema recommends |
I think using allOf would mean that we still need to copy the whole schema. This is also due to to having additionalProperties: False as our meta-schema would have to have all the properties too (or at least the ones we use) for it to pass. So we get bad error messages (these consist of basically saying that all of the schemas did not pass without saying why) without any gains in brevity. My suggestion would be to use JSON merge patch internally. So we would have 2 files. |
Ah, true. All sounds good. |
Pull request #611 now separates draft-04 from the patch we apply to it. It has tests too make sure the patch is applied if it modified. |
These could be added to meta-schema-patch.json, but JSON Schema Draft 4 itself doesn't have titles or descriptions, so it seems acceptable to document the new fields in our documentation. Noting that, before we replace |
[#566] Add meta-schema.json and test using it.
Hi guys we have just implemented devgateway/jocds#14 Wanted to see if there is a plan to include the new metaschema in the Thanks much ! |
@mpostelnicu Great to hear! This should be part of 1.2, but we don't yet have a release schedule for it. |
|
@jpmckinney I've reviewed the linked comment and added my suggestions below. Let me know if you're happy for me to prepare a PR on that basis.
We're doing this as part of 1.2 so this change will go out to consultation.
I assume we'll only add this to version 1.2 onwards.
I can't imagine we'd change the metaschema outside of a minor release of OCDS, but it seems simplest to release the metaschema along with each OCDS release, even if nothing changed.
I suggest that we add a paragraph or sub-heading to https://standard.open-contracting.org/staging/1.2-dev/en/schema/ since that page already describes OCDS's use of JSON Schema.
I think a test can be added to
Deployment is now automated and I think https://github.com/open-contracting/deploy/blob/main/deploy-docs.sh will automatically deploy the meta-schema. If we're only doing this for version 1.2 onwards, we don't need to anything historically.
This was resolved in #611 |
Yes, ready for PR. |
Originally reported via email by @devgateway, quoted below:
The
$schema
section should define the meta-schema of the current schema. This is used by validation tools that comply with the standard, to correctly identify entities that are part of the schema. Further read on the standard page:http://json-schema.org/latest/json-schema-core.html#rfc.section.7
and another third party page linked from json-schema.org, they say it is a good read:
https://spacetelescope.github.io/understanding-json-schema/reference/schema.html
The issue is OCDS defines draft-04 as the reference meta-schema:
"$schema": "http://json-schema.org/draft-04/schema#"
Because extra properties are defined in the OCDS schema files, the line above should read something like this:
"$schema": "http://standard.open-contracting.org/schema/1__1/meta-schema.json"
which should be an extension of draft-04, adding the
deprecated
keyword, and others that were added on top of draft-04.This meta-schema extension should allow:
validating the schema itself against the meta-schema (which is self-validating). For example if someone introduces now a typo in the definition of deprecated for a new deprecated field, say
deprecatedVersoin
instead ofdeprecatedVersion
, this will be spotted easily.make validation tools know what rules to apply for validation, right now they correctly ignore all these extra keywords because
$schema
says it only has entities defined from draft-04 (which in the case of OCDS release files is false).So for me, because I use one of these validating libraries in jOCDS, I need to hack the OCDS schema and either remove
$schema
altogether (which is a problem in itself), or write my own$schema
and define your extra keywords there...The text was updated successfully, but these errors were encountered: