How to watch the progression of a new sos release to the end Linux distribution?

[TrevorBenson]

This goes for any distribution, although my primary focus is really for Red Hat Enterprise Linux ecosystem (Fedora, CentOS Stream, RHEL) and Ubuntu ecosystem. @arif-ali was kind enough to share History for sos-system-data-collection with me in Libera, although this appears to focus on when the process is completed.

I know this is kind of out of scope for the sos project itself, and primarily a concern of each individual distributions. But since both Ubuntu and RHEL contribute heavily to this project I figured this couldn't hurt to ask.

Goal

Partially for general knowledge. However, partially to learn if there is anything "to do" that would ensure a smooth transition between the release being generated and any "first steps" (tickets, etc.) to having the upstream distributions start their processes.

Distributions

Red Hat Enterprise Linux / CentOS Stream Ecosystem

I've seen a bit from the Fedora side on new packages being requested in a bugzilla ticket. For v4.9 Upstream Release Monitoring generated the ticket within 1 hour of the release, before the sos-devel mailing list announcement even is sent, although for sos 4.8.0 it appeared to take around 4 week (8/17/2024 release, 9/13/2024 sos-4.8.0 is available ticket. Also, Fedora 40/41 targets CentOS 10 Stream so it makes sense this would lead to CentOS 10 Stream and eventually RHEL 10.

For RHEL 9 there is no longer a Fedora 39/38 leading into CentOS 9 Stream. Is there a way to track or even submit a request in this case?

For RHEL 8 which no longer has either upstream of CentOS 8 Stream or Fedora, is there a way to track or submit? Or is dnf clean metadata ; dnf check-update sos & History sos-system-data-collection page the only options?

Canonical / Ubuntu Ecosystem

I presume this is based on Debian and the transition from unstable -> testing -> stable repo. Does Ubuntu track "package-X.Y.Z is available" via bugzilla in the same way Fedora does? Besides the page Arif shared, is there another way to track it?

Thanks in advance.

[arif-ali]

I can answer from the Debian/Ubuntu ecosystem, at least since I took over the maintainer role recently. I do plan to document some of this in the wiki, but below is some brief guidelines and some useful links.

First we go through some initial packaging, and initial testing. This boils down to the following tasks

1. lrc
2. lintian
3. reprotest
4. pbuilder
5. sbuild (chroot)
6. sbuild (unshare)

Once this is all successful after any modifications required, we then upload to Debian either directly or get our package sponsored.

• If the package has to be sponsored, then typically it will first appear https://mentors.debian.net/package/sos/, you will see that 4.9.0 is there ready for a Debian Developer to sponsor
• Once the package is sponsored, then the details and the transition and regular updates will appear https://tracker.debian.org/pkg/sos; you can see the old package details at https://tracker.debian.org/pkg/sosreport
  • Noting at the moment we are in a transition phase of going from the sosreport name to sos, so this has not been migrated to testing yet.
  • Note also, we will be able to get our 4.9.0 package sponsored which will still stay in unstable while the sosreport issue hasn't been solved.
• Typically, after 6 to 12 hrs of it being in unstable, ubuntu will auto-sync into the latest dev releases if we're outside feature freeze. Otherwise we have to request a manual sync providing strong reasons for the sync to happen.
• If this has happened, you can track this at https://launchpad.net/ubuntu/+source/sos, a useful link would be top right and looking through the publishing history. Obviously this is now the new package sos, but you can see sosreport package at https://launchpad.net/ubuntu/+source/sosreport, which is currently for the stable releases lower than or equal to 24.10
• Once the package arrives in the proposed pocket, it needs to run autopkgtests, and only once this passes, it will automatically migrate to the release pocket. The results of this can be monitored at sos or sosreport, and by clicking the excuses link if it hasn't migrated.

For ubuntu stable releases, there's a different process.

1. Create a new bug that explains the update based on the SRU template
2. Then if we have upload permissions, then we will build the package for each of the supported releases and upload. There are many links here, but in essence it goes to the Unapproved queue for then the SRU team to approve. Once approved, it goes into the proposed pocket.
3. This will also go through the same autopkgtests to ensure there is no regression
4. As sos has an SRU exception, we have to go through verification that is documented to ensure that we had no regressions. (you'll see on this page the links for the bugs at the bottom each time there's been an SRU)
5. Once this passes then the package is released to the updates pocket.

Note: I am pretty new to the whole process, but once I document this, it will be much more clear. Any questions or comments then happy to answer them

Reference:
SRU == Stable Release Update

[TurboTurtle]

Red Hat Enterprise Linux / CentOS Stream Ecosystem

I can comment on this up to a point, though parts may be outdated as I left RH in mid 2023.

Once an upstream release is cut, internally the team that is responsible for sos creates a tracking ticket. This may or may not be publicly visible, the current Red Hatters on the (upstream) sos team can clarify further. The upstream source tarball is then pulled, and an updated package is built (including any RH-specific patches, if they exist) and submitted to the QE folks that are attached to the internal-to-RH-sos-team, who do their testing and validations.

If faults are found, QE kicks the build back to the development team. Update commits are then made and generally sent to upstream at the same time the same commit is added as a downstream patch to the rpm build; of course exceptions can be made around the timing of all this depending on criticality but from my experience that was very rare. An updated rpm is built that includes the patch(es) and sent back to QE. Rinse and repeat as needed until QE signs off on a particular build.

Alongside this, the release of an sos update is determined by PM and the internal sos team. I cannot speak to the current cadence of RHEL updates for the sos package, or how they do or don't align to RHEL major/minor/z-stream updates. What I can say, however, is that when I was involved in the process, sos minor releases (4.Y.0) were mostly synced to RHEL minor releases, and there was a regular cadence for "z-stream" (4.Y.Z, e.g. 4.8.2) which would release under RH parlance as an "async" update (typically categorized as an RHBA). I don't want to step on any toes and dive into specifics here.

What is important to note though is that nowhere in the process does the Fedora or CentOS Stream release have a direct impact, unlike pretty much every other package that ships in that family of distros. sos is shipped and updated in RHEL in step with what is needed by customers and RH's support organization, without restrictions placed on it by distros "between" upstream (us) and RHEL.

Usually the way to track package releases with RH is to file a customer support case asking for tracking. I'll defer to the RH crew here to offer any alternatives.

[pmoravec]

I can confirm this process in delivering sos package in RHEL is roughly the same as Jake recalls. Depending on various factors, sos RHEL package is shipped in supported RHEL [8-9].Y releases usually a few weeks after the upstream release.