feat!: extend bundling to create another schemas without $id

[derberg]

⏰ BREAKING CHANGE ⏰

@asyncapi/specs package has now different structure. You no longer access specs directly by version name but there are 2 properties:

• schemas - with previous schemas object
• schemasWithoutIds - with new set of schemas that do not contain $id

For more details check out readme updates about $id and examples on how to use the package

---

Fixes #247
Fixes #367

This is a workaround implementation. I'm not proud of it but it is better than the current situation.

Our AsyncAPI JSON Schema is created following JSON Schema best practices and also requirements of https://github.com/hyperjump-io/json-schema-bundle - which means we have beautiful $id and our references are based on these, which makes schema unreadable for humans (like me that completely do not understand $id and $ref concept, but my issues could be ignored and I could learn) and are not understood by some tools (like https://github.com/redhat-developer/vscode-yaml that VSCode YAML autocompletion and validation is based on, and this is a problem for us).

• Why workaround and not something we can solve at above-mentioned bundler level? -> Allow different bundling technique for schemas below draft 2019-09 hyperjump-io/json-schema-bundle#9
• Why not solve this with the above-mentioned YAML plugin managed by RedHat? -> let me answer with question: what assurance do we get it will not break in the future with further contributions to the plugin, or that other tools do it the proper way?

---

• This PR also contains additional generated schemas, under schemas dir, that are alternative, without $id
• New schemas are now used in "All schemas file" that we use in schema store.
• New schemas are exposed on JS/GO packages level
• I tested newly generated schemas with asyncapi file on local, and autocompletion and validation works

[fmvilas]

Just small suggestions.

[jonaslagoni]

LGTM 👍

So my thoughts the current $id and $ref situation is that we have solved the issue in the bundled schemas now, which is perfect 👌

If however, people want to use a single schema, instead of the bundled one, we still support dynamically fetching those schemas remotely. Which I technically think is good, whether there is a use-case for it is another thing 😆 Cause I don't see it, just from a technical standpoint.

So if we don't see that as a "thing", we could remove the $id yea, but I would probably wait and see tbh 🙂

[magicmatatjahu]

Two suggestions :)

[magicmatatjahu]

So if we don't see that as a "thing", we could remove the $id yea, but I would probably wait and see tbh 🙂

I would leave it at that, even if we currently have no use case for that.

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[derberg]

@magicmatatjahu @jonaslagoni the problem is that once I leave $id then parser will not work with my changes because of errors like:

{
message: "can't resolve reference #/definitions/contact from…d http://asyncapi.com/definitions/2.5.0/info.json", 
missingRef: 'http://asyncapi.com/definitions/2.5.0/info.json#/definitions/contact', 
missingSchema: 'http://asyncapi.com/definitions/2.5.0/info.json'
}

because ajv, when it sees $id, wants to resolve $refs against it.

If however, people want to use a single schema, instead of the bundled one, we still support dynamically fetching those schemas remotely

what do you mean by single schema? like https://github.com/asyncapi/spec-json-schemas/blob/master/definitions/2.5.0/channels.json ? the $ref that we use there is absolute so I guess $id is not necessary. Unless you meant something different

omg I don't think I will ever like $id

[jonaslagoni]

@derberg yea you're right, forgot that the $id changes where relative references are looking. For the bundled schemas you need to remove them 🙂

Thought you meant to remove them from the individual definitions.

[derberg]

Thought you meant to remove them from the individual definitions.

Actually, why not? I seriously do not get why we need it there 😆
example:

{
  "type": "object",
  "propertyNames": {
    "type": "string",
    "format": "uri-template",
    "minLength": 1
  },
  "additionalProperties": {
    "$ref": "http://asyncapi.com/definitions/2.5.0/channelItem.json"
  },
  "$schema": "http://json-schema.org/draft-07/schema#",
  "$id": "http://asyncapi.com/definitions/2.5.0/channels.json"
}

if $ref is anyway an absolut URL, what is $id for....asking for a friend 🙏🏼

[magicmatatjahu]

If the schema is single, the $id is not needed, but if it is standalone, it is better to leave it as it determines the id of this single schema.

EDIT: That $id is used to bundle schemas in proper way.

[char0n]

Actually, why not? I seriously do not get why we need it there

$id keyword primary role is to establish JSON Schema identity. It's an important and sadly, very complex concept.

There are three concepts in play:

base URI - is established by resolving resolved $id against retrieval URI
retrieval URI - is URI where the JSON Schema has been retrieved (protocol based or local file system)
$id - is established by resolving the current $id with all parents $id, up to root JSON Schema $id

EDIT: That $id is used to bundle JSON Schemas in proper way.

Exactly. It's only RECOMMENDED practice to use $id when no referencing of JSON Schemas happens, but it's NEEDED to have when referencing JSON Schemas. Simply said, $id gives us ability to uniquely identify the JSON Schema, even if the retrieval URI of the JSON Schema is different from the $id. Example: JSON Schema can be physically served from different HTTP URLs, but if it contains the same $id it's considered to be the same JSON Schema. You want to bundle it just once.

[derberg]

@char0n thanks a lot for detailed explanation

Example: JSON Schema can be physically served from different HTTP URLs, but if it contains the same $id it's considered to be the same JSON Schema. You want to bundle it just once.

now looking at my example above, reference points to http://asyncapi.com/definitions/2.5.0/channelItem.json where $id is different that the $id of schema that there reference comes from

...

ok, I leave $id in source files and remove it from bundled schema

[jonaslagoni]

@derberg any update here, or are you just waiting for reviews? 🙂

[derberg]

ok, so now $id is removed from resulting schema and autocompletion works like a charm when I check manually:

but in Parser it fails with:

Error: can't resolve reference http://asyncapi.com/definitions/2.6.0/schema.json from id #

but....but there is no http://asyncapi.com/definitions/2.6.0/schema.json reference 😭

I thought I need "$id": "http://asyncapi.com/schema-store/2.6.0.json" id in schema root, but that was not a lucky guess

I ran out of ideas, need a break

[derberg]

@jonaslagoni @smoya I changed the package exports, have a look:

• prefix for major release updated
• description of PR updated

[jonaslagoni]

If you dont want 2 major version changes within 3 months, you could merge this into next-major-spec. But just a suggestion 🙂

[derberg]

I do not mind having major every month if that is required 😃 we should not be afraid of major releases in libraries

[smoya]

If you dont want 2 major version changes within 3 months, you could merge this into next-major-spec. But just a suggestion 🙂

I agree. And keep advocating for that over long living PR's targeting master

[smoya]

I do not mind having major every month if that is required 😃 we should not be afraid of major releases in libraries

I'm sure releasing few many major versions in a short period of time can be taken as negative because few reasons. Completely valid for a new discussion but still worth to share IMHO:

• Can create the perception of instability, like the package is not mature enough.
• Contributors can feel disconnected from the repo when realizing few major versions happened without them to even realize.
• Annoying users to update too many times in order to keep up with big changes.

Having said that, its up to you to do one thing or another atm

[derberg]

We have March now, releasing major. Next maybe, I say maybe, will be in June. So we have at least 3 months different. This is not often. Let us not forget it is not a big project. This is super simple package, that only exposes json files, not parser or any complicated beast.

Can create the perception of instability, like the package is not mature enough

the person who has no sins should be the first to throw a stone 😄

have you ever "rejected" to use a package that has many major releases?

• we use Jest everywhere (current release - 29)
• not to mention react
• I can go one and one

anyway, thanks for approval 🙏🏼 😄

[derberg]

@fmvilas I need your approval as you requested changes in the past

@dalelane @char0n have a look as we will release major here

[derberg]

@fmvilas I need your approval as you requested changes in the past

@dalelane @char0n have a look as we will release major here

[derberg]

@smoya please approve again, I had to solve merge conflict with readme cause by new breaking changes section. No more changes

[derberg]

@smoya @fmvilas one approve needed

[sonarcloud]

Kudos, SonarCloud Quality Gate passed!   

0 Bugs
0 Vulnerabilities
0 Security Hotspots
0 Code Smells

No Coverage information
0.0% Duplication

[derberg]

/rtm

[asyncapi-bot]

🎉 This PR is included in version 4.3.0 🎉

The release is available on:

• npm package (@latest dist-tag)
• GitHub release

Your semantic-release bot 📦🚀