Add package download retry in anaconda-iso to avoid The requested URL returned error: 503

[henrywang]

Recently, I found a lot of anaconda-iso failures, like source/org.osbuild.curl (org.osbuild.curl): WARNING: failed to download http://ftp-osl.osuosl.org/pub/centos-stream/9-stream/AppStream/aarch64/os/Packages/mesa-libEGL-24.2.8-1.el9.aarch64.rpm: The requested URL returned error: 503. Can we add something like retry to avoid this kind of error? Thanks.
For example:
https://artifacts.osci.redhat.com/testing-farm/5829666b-7144-4c15-bdc2-cfaae71dd0f1/
https://artifacts.osci.redhat.com/testing-farm/58eeb608-fa95-40e5-952b-41154ebb9e6c
https://artifacts.osci.redhat.com/testing-farm/1daee862-99fa-4e0f-8948-ac71aa6463b8/
https://artifacts.osci.redhat.com/testing-farm/c75f6d84-5089-46c1-ad3d-b56107493fd7/
https://artifacts.osci.redhat.com/testing-farm/d73175e8-a849-4f11-adcc-dddc0b21d46d/
https://artifacts.osci.redhat.com/testing-farm/b6af1d03-6a42-4b0d-b764-dc215c47fec7/

[mvo5]

Thanks for the problem report. We noted this as well and we believe that switching from "plain" libcurl to librepo as the download backend will fix the problem. Bib has supoort for this since a couple of weeks and we use it in our own CI since and it seems to be working quite well.

However (and the reason why we have not switched to librepo by default) - we do not have good coverage for subscribed content in our own integration tests. If we have coverage for building isos from subscribed content in the downstream tests that would be great and then I can undraft #840

[ondrejbudai]

@henrywang actually, I think we can use your help here. Can you test that --use-librepo works fine with RHEL-based anaconda-iso (when subscriptions are involved). If you could somehow verify it in your team's testing pipelines, we can then switch --use-librepo to be on by default, and this would fix the CentOS Issues. Thanks! ❤

[henrywang]

Hi @ondrejbudai, @mvo5 , Sure, I can run test with --use-librepo. The only subscribed content should be RHEL 9.5. I'll run test with rhel-bootc:9.5 image. Let's see what happens. Thanks!