Sync with upstream version 0.21.0-4 #131
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Since leapp's RHUI mechanism filters setups based on the architecture of the source system, it was not possible to upgrade of ARM-based RHEL systems on Alibaba cloud as there were no ARM entries in RHUI_SETUPS. This patch adds these entries, making it possible for EL 8 -> 9 upgrades of ARM systems on Alibaba cloud. (cherry picked from commit fbc38d4)
in releases where we do not have any obsoleted keys, we still had to define an entry (with an empty list), as otherwise the code would fail instead, we can catch the KeyError and carry on as nothing happened (cherry picked from commit 7e0fb44)
Previously we temporarily used the RHEL 9 x86_64 prod cert for others archs it was missing completely. Jira: OAMG-11138 (cherry picked from commit 9f2f172)
Focus was on making the CI and GitHub actions work after the default branch was switched from master to main. See: OAMG-4907 (cherry picked from commit 9d49f46)
New version of Pylint have the rule for checking of positional arguments - complaining when more than 4 positional arguments exists. We do not want to refactor the code to make it happy and the default value cannot be set right now - that's planned for future Pylint versions at this moment. So excluding this rule. For more info: * https://pylint.readthedocs.io/en/latest/user_guide/messages/refactor/too-many-positional-arguments.html (cherry picked from commit 88e13fb)
pam_userdb module changed its backend database technology from lidb to gdbm for RHEL10. This requires a set of leapp actors to perform the database migration automatically when upgrading to RHEL10: * ScanPamUserDB takes care of scanning the PAM service folder to detect whether pam_userdb is used and the location of the database in use. This information is stored in a model. * CheckPamUserDB checks the databases reported by ScanPamUserDB and prints a report about them. * ConvertPamUserDB checks the databases reported by ScanPamUserDB and converts them to GDBM format. * RemoveOldPamUserDB checks the databases reported by ScanPamUserDB and removes them. All these actors include unit-tests. Finally, there's also a spec file change to add `libdb-utils` dependency as it is required to convert pam_userdb databases from BerkeleyDB to GDBM. Signed-off-by: Iker Pedrosa <[email protected]> (cherry picked from commit 658700d)
As mirror.centos.org is dead, replace mirrorlist with baseurl pointing to vault.centos.org in utils/container-builds/Containerfile.centos7. (cherry picked from commit d6e57ee)
The missing `leapp.reporting.Report` class is added to kernelcmdlineconfig actor `produces` tuple. (cherry picked from commit b997e4e)
…are listed Instead of erroring out when grubby lists multiple entries for the default kernel, always use the `args=` and `root=` from the first one and create a post-upgrade report. The report instruct user to ensure those are the correct ones or to correct them. This can happen, for example, if MAKEDEBUG=yes is set in /etc/sysconfing/kernel. Jira: RHEL-46911 (cherry picked from commit c2c96af)
(cherry picked from commit 053137c)
(cherry picked from commit d3ebc99)
* The main reason for this change is to improve performance and reduce flood of logs for the content that does not seem to be important to check for the upgrade process. The directory has been relatively recently added to ca-certificates rpm on EL 9+ systems mostly to improve performance of OpenSSL and the content does not seem to be important for the IPU process. The high number of files takes too much time to evaluate and causes flood of logs that are not important. This is updated solution that we drop originally: 60f500e (cherry picked from commit a147938)
On systems with large disks (e.g. 16TB) with lots of free space, leapp might attemt to create files larger than the max file size of the underlying FS. Attempting to create such large files causes leapp to crash. This patch caps the max image size to 1TB, based on empirical evidence that more free space is not needed for the upgrade RPM transaction. Jira-ref: RHEL-57064 (cherry picked from commit cef2825)
This error occurs when repo file has invalid definition, specifically when the 'name' entry of the config files is invalid. Also add tests. Jira: RHEL-19249 (cherry picked from commit ec07824)
(cherry picked from commit f84c6f8)
Modify the StorageInfo model to include path and name of the parent device. Use StorageScanner to collect this information. Morover fix lsblk test, there should be a full device path in "lsblk -pbnr" output (just names were used in the original test). (cherry picked from commit 03fc674)
Add LuksScanner actor that runs 'cryptsetup luksDump' for all 'crypt' from lsblk output. The output is then parsed and filled into LuksDump and LuksToken models. The LuksDump model contains information about LUKS version, device UUID, corresponding device path, name of the backing device (which contains the LUKS header) and a list of LuksToken models. LuksToken model represents a token associated with the given LUKS device. It contains token ID, IDs of associated keyslot and token type. If the token type is "clevis", we use "clevis luks list" command to determine the clevis-specific subtype and append it to the token name. E.g. if there is a "clevis" token and "clevis luks list" returns "tpm2", the token type will be "clevis-tpm2". (cherry picked from commit 266c249)
So far, upgrades with encrypted drives were not supported. Encrypted drives require interactively typing unlock passphrases, which is not suitable for automatic upgrades using Leapp. We add a feature, where systems with all drives configured with automatic unlock method can be upgraded. Currently, we only support drives configured with Clevis/TPM2 token, because networking is not configured during Leapp upgrade (excluding NBDE). We consume LuksDumps message to decide whether the upgrade process should be inhibited. If there is at least one LUKS2 device without Clevis TPM2 binding, we inhibit the upgrade because we cannot tell if the device is not a part of a more complex storage stack and the failure to unlock the device migt cause boot problem. Co-authored-by: Petr Stodůlka <[email protected]> (cherry picked from commit ad241f7)
The actor nowadays does more then just inhibiting the upgrade when LUKS is detected. Let's rename it to respect current behaviour. (cherry picked from commit 8e5fe75)
First resource limit is maximum number of open file descriptors limit, second one being limit for maximum writable file size. Plus add unit tests. Resolves: RHEL-26459 and RHEL-16881 (cherry picked from commit 5e6d176)
Leapp writes .link files to prevent interfaces being renamed after booting to post-upgrade system. This patch adds a less error-prone approach that uses net.naming-scheme kernel param. The naming-scheme tells udev what hardware properties to use when composing a device name. Moreover, possible values of this parameter are coarse-grained "profiles", that tell udev to behave as if it did on RHEL8.0. The functionality is enabled by setting LEAPP_USE_NET_NAMING_SCHEME environmental variable to 1. If the feature is enabled, the .link file generation is disabled. A kernel parameter `net.naming-scheme=` is added to the upgrade boot entry and the post-upgrade entry. The value of the parameter will be `rhel-<source_major>.0`. Note that the minor source version is *not used*. Using also source major version instead of 0 causes the device names to change slightly, so we use 0. Moreover, an extra RPM named `rhel-net-naming-sysattrs` is installed to the target system and target userspace container. The RPM provides definitions of the "profiles" for net.naming-scheme. The feature is available only for 8>9 and higher. Attempting to upgrade 7>8 with LEAPP_USE_NET_NAMING_SCHEME=1 will ignore the value of LEAPP_USE_NET_NAMING_SCHEME. Add a possibility to use the net.naming-scheme cmdline argument to make immutable network interface names during the upgrade. The feature can be used only for 8>9 upgrades and higher. To enable the feature, use LEAPP_USE_NET_NAMING_SCHEME=1. Jira-ref: RHEL-23473 (cherry picked from commit e1bdf2c)
(cherry picked from commit b4b5354)
(cherry picked from commit e43a892)
The fdisk output can contain warning msgs when a partition is not
aligned on physical sector boundary, like:
Partition 4 does not start on physical sector boundary.
We know that in case of MBR the line we expect to parse always
starts with canonical path. So let's skip all lines which does
not start with '/'.
jira: https://issues.redhat.com/browse/RHEL-50947
(cherry picked from commit 0bf07d1)
Address issue with ARM system upgrades from RHEL 8 to RHEL 9.5+ caused by GRUB bootloader incompatibility with newer kernels. When attempting to load the RHEL 9.5+ kernel using the RHEL 8 bootloader, the upgrade process halts due to a boot crash. JIRA: 41193 (cherry picked from commit abcf7a5)
The `/usr/share/ruby/irb` path is a symlink in RHEL 9, but a regular directory in RHEL 10. This puts us back in line with RHEL 8 and Fedora in terms of the path's file type regarding the rubygem-irb package. Since this was not handled on RPM level, handle it as actor again. This was copied and adjusted from same-named el8->el9 actor. We do not care about the validity or target of the symlink, we just remove it to allow DNF create the correct directory on upgrade. Without this workaround, the upgrade will fail in transaction test with reports of file conflicts on the directory path. Users should not expect to ever retain anything in this directory. (cherry picked from commit 866a4b9)
In RHEL 10, the directory is a regular directory again. The 2 actors are separate over creating a common solution for both. Expand in the docstring on the reason for the el8->el9 actor to differentiate them apart. (cherry picked from commit 81a3297)
Update data files to a newer version. (cherry picked from commit 2dc7efa)
The leapp actors configuration feature is present since leapp-framework 6.0. Update the dependencies to ensure the correct version of the framework is installed on the system. Also, leapp requirements have been updated - requiring python3-PyYAML as it requires YAML parser, bumping leapp-framework-dependencies to 6. Address the change in leapp-deps metapackage to satisfy leapp dependencies during the upgrade process. (cherry picked from commit 89afbe8)
Add additional build steps to the specfile that create the actor configuration directory. The directory is owned by the package, so it gets removed when the user uninstalls leapp. Also prepared some comment lines for future when we will want to include some configuration files as part of the rpm. (cherry picked from commit 36b93e4)
We have several .gitkeep files in the repo as we want to have some directories present in git however these directories are empty otherwise. This is common hack to achieve this, but we do not want to have these files really in the resulting RPMs. So we just remove them. (cherry picked from commit 87db66c)
Load actor configuration when running `leapp upgrade` or `leapp preupgrade`. The configuration is loaded, saved to leapp's DB, and remains available to all actors via framework's global variable. (cherry picked from commit 140a0bb)
Introduce a common configuration definition for RHUI related decisions. The configuration has an atomic nature - if the user wants to overwrite leapp's decisions, he/she must overwrite all of them. Essentially, all fields of the RHUI_SETUPS cloud map entry can be configured. Almost no non-empty defaults are provided, as no reasonable defaults can be given. This is due to all setup parameters are different from provider to provider. Therefore, default values are empty values, so that it can later be detected by an actor whether all fields of the RHUI config has been filled. Jira ref: RHEL-56251 (cherry picked from commit f3d3832)
Extend the check_rhui actor to read user-provided RHUI configuration. If the provided configuration values say that the user wants to overrwrite leapp's decisions, then the patch checks whether all values are provided. If so, corresponding RHUIInfo message is produced. The only implemented safe-guards are those that prevent the user from accidentaly specifying a non-existing file to be copied into the scrach container during us preparing to download target userspace content. If the user provides only some of the configuration values the upgrade is terminated early with an error, providing quick feedback about misconfiguration. The patch has been designed to allow development of upgrades on previously unknown clouds (clouds without an entry in RHUI_SETUPS). Jira ref: RHEL-56251 (cherry picked from commit a03e8e5)
Extend the CurrentActorMocked class to accept a `config` value, allowing developers to mock actors that rely on configuration. A library function `_make_default_config` is also introduced, allowing to instantiate default configs from config schemas. (cherry picked from commit a206a7f)
We copy files into the target userspace when setting up target repository content. If this file is named equally as some of the files installed by the target RHUI client installed during early phases of target userspace setup process, we would delete it in cleanup. Therefore, if we copy a repofile named /etc/yum.repos.d/X.repo and the target client also owns a file /etc/yum.repos.d/X.repo, we would remove it, making the container loose access to target content. This patch prevents us from blindly deleting files, keeping files that are owned by some RPM (usually that would be the target RHUI client). (cherry picked from commit 0147bc2)
* Add product certificates for RHEL 9.6
* Introduce upgrade path 8.10 -> 9.6
* Drop IPUs related to EL 8.8 and 9.2
* This will not be supported in this release.
* Keeping for now still IPU 8.10 -> 9.5 as it is a fresh release
so it has a value for us to run tests there. We will drop it
later during this lifecycle (CTC-2?).
* Drop EL 8.8 from the list of supported versions
* Update tests in packit
* Note that tests for 9.6 could be failing for a while until
composes are created.
jira: RHEL-67621
(cherry picked from commit c2f2895)
The directory should be provided by the framework. leapp-repository should provide only a content inside if any present. (cherry picked from commit f50e347)
(cherry picked from commit 9c07443)
This commit enables the use of net.naming-scheme for 8>9 upgrades by default. The previously used environmental variablel LEAPP_USE_NET_NAMING_SCHEMES is replaced with LEAPP_DISABLE_NET_NAMING_SCHEMES with inverse semantics. (cherry picked from commit 3c3421a)
Thank you for contributing to the Leapp project!Please note that every PR needs to comply with the Leapp Guidelines and must pass all tests in order to be mergeable.
Packit will automatically schedule regression tests for this PR's build and latest upstream leapp build.
Note that first time contributors cannot run tests automatically - they need to be started by a reviewer. It is possible to schedule specific on-demand tests as well. Currently 2 test sets are supported,
See other labels for particular jobs defined in the Please open ticket in case you experience technical problem with the CI. (RH internal only) Note: In case there are problems with tests not being triggered automatically on new PR/commit or pending for a long time, please contact leapp-infra. |
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
directory-hashdir to speedup the upgrade process and clean logs