containers: Remove the need for yum update in Dockerfile
#1043
Conversation
|
Verifying the imaginary situation where matching packages are absent in the configured repositories. Let's say 18.2.4 is released with updated package repositories but container images are not yet built with the updated packages: Now we attempt to install development packages: It works. |
anoopcs9
added
extended-review
|
Thanks for the detailed explanation. However, I think it's based on a hypothetical practice that the ceph project doesn't follow. Let me try to explain: the That would be fine if ceph rpm repos were "additive". However, that's not how ceph organizes the repos. Let's take reef for example. On download.ceph.com there are dirs named for ceph releases. For example, "rpm-18.2.4", "rpm-18.2.2", and so on. If you look in each of these you will see that they only contain packages with a matching version number. rpm-18.2.4 doesn't contain both 18.2.4 and 18.2.2 versions. Then there's a link to the latest release "rpm-reef" (it's not obvious it's a symlink from the webserver but I've been told that's what it is). If you look at a release container image you'll see that dnf config points at So, for example, when the image is built with 18.2.2 but in the mean time rpm-reef becomes rpm-18.2.4 dnf won't even see any 18.2.2 rpms in the repo. I don't think dnf can handle the case the way you described it... but please correct me if I'm wrong. We could consider asking the ceph team to build release container images with dnf configured with Let me know if you think my concerns are valid or you have any other thoughts. |
The situation I created above only had libcephfs-devel-18.2.4 in the repositories(barring the other older version from ganesha repo). libcephfs-devel is initially not present with the base container image.
If I am not wrong that's how RPM repositories are found to be structured in general.
Yes, my artificial situation above explicitly displays these baseurl values.
DNF is intelligent enough to find an updated version of a package as soon as it refreshes the repositories invalidating outdated repodata. Thus in the situation I created above it finds libcephfs-devel-18.2.4 from updated rpm-reef(pointing at rpm-18.2.4) and subsequently decides to update every other inter-dependent packages which are available from the repository link currently configured with rpm-reef.
That would be ideal but definitely not a blocker for us in this situation.
I hope I filled in the gaps from my previous explanation. Accordingly I think the changes(or in other words DNF 😉 ) can handle the anticipated problem during release time. |
|
OK, I'm not entirely convinced but I'm convinced enough to run the experiment. :-) |
|
@Mergifyio rebase |
✅ Branch has been successfully rebased |
phlogistonjohn
force-pushed
the
containers-rm-yum-update
branch
from
65f00b8 to
664d1b0
Compare
Can you point out where this best practice recommendation comes from? |
May be I went a step ahead and bent the wording from "desirable" to "best practice" out of an offline discussion involving people associated with building containers (ceph in this context). But I tend to agree that it becomes unpredictable with a moving target like CentOS Stream, where updates to any one of the dependencies for the already built or installed packages can affect the creation of new container images out of it. Even if we don't generalize, I would still suggest removing the blind update step, provided we can achieve the goal (of overcoming the anticipated problem during release time) in its absence. What do you think? |
I don't really have a strong opinion about it. However not all container-images get rebuilt regularly and are therefore missing security/bugfix updates. For Ceph-CSI we receive the occasional requests to rebuild images with updates installed. I understand the position from image producers to have fully tested images, but I also understand the users, as they want fewer known bugs/CVEs in the images 🤷♂️ |
Here in this case, we don't have any official container images meant for distribution as part of each release. This is just a matter of creating CI (short-lived) containers where we can be less bothered by security or bugfix updates to packages. |
|
@Mergifyio refresh |
✅ Pull request refreshed |
|
@Mergifyio rebase |
✅ Nothing to do for rebase action |
|
@Mergifyio rebase |
✅ Branch has been successfully rebased |
anoopcs9
force-pushed
the
containers-rm-yum-update
branch
2 times, most recently
from
763f661 to
36ab7c5
Compare
|
Another demonstration of how dnf overcomes the anticipated problem with recently released v17.2.8. Please note that v17.2.8 doesn't have el8 container image compared to previous v17.2.7 where it didn't have an el9 variant. Therefore we manipulate an el9 container image for v17.2.7 by downgrading packages within v17.2.8. In effect we have a v17.2.7 el9 container. Now: This ends up updating every ceph related packages to the latest v17.2.8 because of their inter dependency. |
|
I am not sure what you are trying to do, or why... If you want to use Ceph v17.2.7, would you not use the As CentOS 8 is not maintained anymore (and neither is Ceph v17.2.7), Ceph moved it's base OS to CentOS Stream 9. |
In the absence of
Our CI jobs doesn't depend on specific minor updates for a particular ceph release rather uses the more generic major release tag(v17, v18, v19 etc.). The situation is a special case when container images are not yet available for a minor update but RPM repositories are already updated to point at latest minor update. #1043 (comment) has some pointers on why this was considered to be a problem.
Yes, and that's why I highlighted the explanation with a note in #1043 (comment). |
|
@Mergifyio rebase |
This reverts commit df3b6a9.
This reverts commit d4eac7d.
In general it is not desirable to blindly update packages as a whole while building another from a base container image. Historically this step was required due to the introduction of version specific installation[1] of packages i.e, we extract the package version that comes with the base container image and try to install the matching development libraries which might be unavailable close to a new release happening in upstream. In order to overcome this short gap we came up with the idea of `yum update`[2] to fetch whatever is the latest and then extract the version for further installation of development libraries. This seemed to work until we discovered a different issue where updated versions for particular dependencies are pushed to standard repositories causing problems[3] during `yum update`. Ceph repositories(and packages) are now more robust and DNF is capable of handling such situations to figure out the new/updated versions for packages even if a match is not found with the already installed package versions. Ideally it can never be the case that matching packages for each version are missing from a particular repository directory(only the links to the directories is supposed to change). Thus in our best interest we avoid running `yum update`. [1] ceph#331 [2] ceph#510 [3] ceph#1038 Signed-off-by: Anoop C S <[email protected]>
Now that we have removed the `yum update` step, it doesn't make sense to install the matching development packages based on the version already present with the base container image. This is due to the fact that their availability is not always guaranteed. Instead leave it up to DNF to figure out if higher versions are available with the repositories. Signed-off-by: Anoop C S <[email protected]>
✅ Branch has been successfully rebased |
phlogistonjohn
force-pushed
the
containers-rm-yum-update
branch
from
36ab7c5 to
adba452
Compare
anoopcs9
removed
the
extended-review
In general it is not desirable to blindly update packages as a whole while building another from a base container image.
Historically this step was required due to the introduction of version specific installation(#331) of packages i.e, we extract the package version that comes with the base container image and try to install the matching development libraries which might be unavailable close to a new release happening in upstream. In order to overcome this short gap we came up with the idea of
yum update(#510) to fetch whatever is the latest and then extract the version for further installation of development libraries.This seemed to work until we discovered a different issue where updated versions for particular dependencies are pushed to standard repositories causing problems(#1038) during
yum update.Ceph repositories(and packages) are now more robust and DNF is capable of handling such situations to figure out the new/updated versions for packages even if a match is not found with the already installed package versions. Ideally it can never be the case that matching packages for each version are missing from a particular repository directory(only the links to the directories is supposed to change).
Thus in our best interest we avoid running
yum update.