Establishing a breaking change policy for groups

[Perksey]

The strongly-typed enum groups which were "standardised" in #335 and have been heavily iterated upon for correctness since are garnering more use as time goes on. Google (for ANGLE) and Arm (for a debugging/API trace tool: more info).

Given Khronos do not use the group information directly themselves, they have often gone forgotten about and whatever a incoming community contribution says, goes. I think given the increasing importance of these groups we should be wary of breaking changes.

In bindings generated from the gl.xml, the groups may be used to define more friendly API signatures. A good example from the Silk.NET library can be found here. By changing group information in gl.xml, this causes a break in that library. We have come to accept this, but it's important Khronos knows that implication.

One core tenet of the groups is correctness. They exist to provide information on what enums are valid for a particular parameter. As such, I do think we need some way of changing a parameter to a more correct one but how this looks from a breaking change policy perspective, I don't know. The thing is no group change, even if we did use enum { ... }s in C, would be breaking due to how C works.

But yeah, this issue mostly serves as a discussion as I don't know what a good (or any) solution is to this.

Thoughts?

• @joakim-arm @NogginBops @3b @SunSerega @devel0 @tsuoranta @Lucretia @PJB3005 @tksuoran @aloucks @pablode @tkaaad97 @StailGot @mode777 @luca-piccioni (as people who have authored PRs that are exclusively regarding groups in the past)
• @null77 (ANGLE stakeholder)
• @Dav1dde (GLAD)
• @HurricanKai @ThomasMiz (ccs from the Silk.NET project)

[Lucretia]

The groups are so important for strongly typed languages, like Ada, where we can have enums which enclose the allowed values.

[NogginBops]

You forgot to give @frederikja163 a ping.

Personally I do think that because the enum groupings are very loose as they are today saying that changes to them cannot be made feels like a very bad decision. It will lock the groups into the usability they have today, which albeit useful are not as useful as they could conceptually be.

In an ideal case I would want every enum function argument in gl.xml to be annotated with an enum group that specifies all of the enums that could be valid for that argument (given all extensions and newest OpenGL version).
I'm not sure how possible that is, as I do not know how many unique enum sets the API uses (but my guess is that it's quite a few and that it might not be feasible to create meaningful names for all of them). So some compromises will probably have to be made.

I feel like the best course of action here would be to start defining what some of the different enum groups are "supposed" to mean and slowly expand these definitions and make sure they are used "correctly".
In my mind that will mean many breaking changes to the groups over quite a long period of time.

If someone wants to have a "stable" version they can make a fork/copy of gl.xml and update that when they can bring in breaking changes.

And with the frequency of PRs relating to enum groups not being that high I think that reviewing them on a case by case basis is the smallest "friction / progress" ratio.

[Lucretia]

It is possible to do. Most people forget or just don't know that these groupings existed in the OpenGL spec files years ago, because SGI used to use them to generate Ada bindings to OpenGL. Any talk of OpenGL is a C API is arrogantly forgetting other languages that use this information and there are many, they need to be maintained.

[oddhack]

If you folks can get together and figure out a governance mechanism for these annotations that we can incorporate in gl.xml without much churn (in particular, without affecting the C headers), that's great. But it's not arrogance to say that Khronos only defines a C API for OpenGL, and we don't have time or motivation for active involvement in other language bindings. That goes back more than 20 years to outside projects like Arcane's Java bindings and the FORTRAN 90 bindings developed by someone at NIST.

Standardization is almost exactly the opposite of "move fast and break things". It moves excruciatingly slowly at times because there are so many stakeholders, and breaking changes are very painful to downstream consumers.

[Lucretia]

You don't have time to actively design things properly? Because that's really what this is about and always has been. C is literally one of the worst languages around, just defining everything in C macros is incredibly lazy. and it is arrogance to think that only C exists and not to think about how do things right.

These aren't breaking changes, a parameter has a set of inputs, so give them a name! This obviously applies less to bitmasks and these are separated from the enum's in the xml files.

[Perksey]

@Lucretia They would be breaking changes if we wanted to change the group a parameter takes, this is the issue. We can add more groups no problem, but removing groups or changing the param or proto group attributes would cause a breaking change downstream.

Also, OpenGL has been around since the 90s and can't do anything much about decisions made back then, such as the C standard to use. C90 (with enums) had just came out two years ago by the time the first version of OpenGL launched, so it likely wasn't prevalent in industry standards by that time. Alternative C headers are welcome to make these decisions for themselves, but the Khronos official ones will likely stay the same as they always have been - anything more will break someone.

To this end, Khronos only cares about the official C headers and the only maintenance work or considerations done by them will be towards these. Groups, lengths, and everything else are optional extras that were carried over from the original gl.spec -> gl.xml transition. The OpenGL WG could always change this, of course, but given the majority of the people there are using C/C++ like most graphics applications, fat chance.

---

If you folks can get together and figure out a governance mechanism for these annotations that we can incorporate in gl.xml without much churn (in particular, without affecting the C headers), that's great.

I think the problem is that all the improvements to the groups to date have been incremental, and as such changes to one group to accommodate one function may invalidate its use in another. I don't want breaking changes, but at the moment some groups are fundamentally invalid/wrong so we need to break for correctness. We'd need to look over the groups as a whole to validate them before we can commit to not breaking I think, which I think is a mammoth task. But if community projects are willing to make efforts in this area I'm all for it!

[SunSerega]

@Perksey

try and establish a sane baseline _[1]

Ok, was mostly walking past this because I agree with:

Personally I do think that because the enum groupings are very loose as they are today saying that changes to them cannot be made feels like a very bad decision _[2]

I am very afraid the breaking change policy will limit future improvements.
But if I understood that message right - you want to limit changes Only after making a big push, possibly with a lot more incompatibility downstream than usual. Now that is something I can get behind.

However, the path is still too abstract. For instance, how would you define

a point where we're happy with what we have _[1]

? I think that idea of having a "point" is absurd here. This can only work, like many other fixes to this huge registry - incrementally.

---

From my experience so far - once in a while, there is some small change, which no one expected to be breaking, but it still breaks my bindings in some (usually easy to fix) way. Both here and in the case of OpenCL.
So, for the XML consumer, having a local copy of data from XML and only pulling changes from upstream when the said XML consumer is ready - is already a given, implicitly.

To start, we can go further and explicitly say:
There is a guarantee to the XML consumer that the useful info from the previous version of XML will remain in the new version, as long as it was not false (one way or another).
But, otherwise, the XML consumer is responsible to check XML changes and how they apply to their downstream thing. And to then change their data-scrappers/code-generators reasonably, to accept how the needed info is stored in the new version of XML.

That is effectively how things worked so far.
But this way we can have a strong base guarantee, which XML consumers can rely on.
And at the same time, XML maintains a broad range of available changes.

Then, we can build on this base guarantee, in multiple different directions at the same time.
For instance, after #534 we can say with confidence that group=... on <proto> at <param> will only ever refer to a grouping of enum values.