A package might not be upgraded if the same package have multiple version available in the channels

[lucagiove]

Original issue on launchpad: Upgrade fails if a required package need to be installed but has multiple versions of it in channels

Issue happens with smart 1.4.1 and patch for #1086888 applied
Issue happens by calling "smart upgrade"

We encountered the issue while upgrading packages from multiple channels where some packages where multiple times within channels but with different versions. In that case transaction was calculated and after confirmation of changes smart interrupted because a package was missing, but that package was included within channels.

When we removed the packages which were not unique upgrade did work correctly.

After debugging and finding and fixing a quite obvious bug this also did not happen anymore.

After understanding what the patched code actually should do and did before, I would say the issue happens whenever the changeset contains a package to be installed, which was not present on the system before and moreover had no installed package which would be obsoleted by it, and then only, when the channel holds multiple versions of this package.

The bug is in transaction.py in method _upgrade()

Here is the code in question:

    for pkg in changeset.keys():

        op = changeset.get(pkg)
        if (op and op != origchangeset.get(pkg) and
            pkg not in locked and pkg not in lockedstates):

            try:
                cs = changeset.copy()
                lk = locked.copy()
                if op is REMOVE:
                    self._install(pkg, cs, lk, None, depth)
                elif op is INSTALL:
                    self._remove(pkg, cs, lk, None, depth)
            except Failed, e:
                pass
            else:
                csweight = getweight(cs)
                if csweight < weight:
                    weight = csweight
                    changeset.setState(cs)

I see no chance that origchangeset ever could not be empty. So I do not really understand the purpose of this code.
Probably this could be removed completely.

The bug here is that when origchangeset is always empty, the operation compared with "op" is always "None", so _remove()/_install() will always happen here unless the package is not locked or statelocked. A package is only statelocked when it was installed in the system before the upgrade procedure took place and upgraded only due to _install() here in the _upgrade() procedure, or when it has been required by another package and it is obvious that there might not be another possibility but to install this package. The lock is not set when there are multiple versions of the same package in the channels.

In our case because of this it removed a package from transaction set, which caused that the transaction set then was not complete anymore and the upgrade failed.

Our setup:

channel configuration:

root@upgrade_3:~/local # smart channel --show
smart channel --disable [r8-2]
type = rpm-md
baseurl = http://192.168.58.129/repo/qa/2.5/r8/2

[rpm-sys]
type = rpm-sys
name = RPM System

[custom-r8-3]
type = rpm-md
baseurl = http://192.168.58.129/repo/qa/2.5/r8/3

[r6-4]
type = rpm-md
baseurl = http://192.168.58.129/repo/qa/2.5/r6/4

[2.5-base]
type = rpm-md
name = 2.5 base
baseurl = http://repository/packages/utm-2.5/base

does fail:

root@upgrade_3:~/local # smart upgrade --explain efw-endian-client

Computing transaction...[000] _upgrade()
 [001] _install(efw-endian-client-4:2.9.20-0.endian30@noarch)
  [002] _remove(efw-endian-client-2:2.4.30-0.endian30@noarch)
  [002] _pending()
   [003] _install(jobsengine-2.9.23-1.endian4@i586)
    [004] _install(python-setproctitle-1.1-0.endian1@i586)
    [004] _pending()
   [003] _install(jobsengine-2.9.22-1.endian4@i586)
    [004] _install(python-setproctitle-1.1-0.endian1@i586)
    [004] _pending()
   [003] _install(jobsengine-2.9.19-1.endian4@i586)
    [004] _install(python-setproctitle-1.1-0.endian1@i586)
    [004] _pending()
   [003] _install(jobsengine-2.9.18-1.endian4@i586)
    [004] _install(python-setproctitle-1.1-0.endian1@i586)
    [004] _pending()
 [001] _install(efw-endian-client-4:2.9.19-0.endian30@noarch)
  [002] _remove(efw-endian-client-4:2.9.20-0.endian30@noarch)
  [002] _pending()
 [001] _install(efw-endian-client-4:2.9.18-0.endian30@noarch)
  [002] _remove(efw-endian-client-4:2.9.20-0.endian30@noarch)
  [002] _pending()
 [001] _install(efw-endian-client-4:2.9.13-0.endian30@noarch)
  [002] _remove(efw-endian-client-4:2.9.20-0.endian30@noarch)
  [002] _pending()
 [001] _remove(jobsengine-2.9.23-1.endian4@i586)
  [002] _pending()
   [003] _install(jobsengine-2.9.19-1.endian4@i586)
    [004] _pending()
   [003] _install(jobsengine-2.9.18-1.endian4@i586)
    [004] _pending()
   [003] _install(jobsengine-2.9.22-1.endian4@i586)
    [004] _pending()
 [001] _remove(python-setproctitle-1.1-0.endian1@i586)
  [002] _pending()

Upgrading packages (1):
  efw-endian-client-4:2.9.20-0.endian30@noarch [custom-r8-3]
    Upgrades:
      efw-endian-client-2:2.4.30-0.endian30@noarch (upgraded)

40.9kB will be used.

Confirm changes? (Y/n):

Committing transaction...
error: efw-endian-client-2.9.20-0.endian30 requires jobsengine >= 2.6.6

content of the channels is:

r6-4:

root@stan:/var/www/repo/qa/2.5/r6/4 # ls -l
total 14880
-rw-rw-r-- 1 clamav 1000 24385 Sep 13 2012 efw-dhcpd-2.9.1-0.endian7.noarch.rpm
-rw-r--r-- 1 clamav 1000 85706 Mar 12 10:18 efw-guilib-2.9.7-0.endian4.noarch.rpm
-rw-rw-r-- 1 clamav 1000 1484559 Mar 8 13:20 efw-hotspot-2.9.54-1.endian9.noarch.rpm
-rw-r--r-- 1 clamav 1000 1485032 Mar 14 11:51 efw-hotspot-2.9.55-1.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 1485629 Mar 20 16:25 efw-hotspot-2.9.56-1.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 1485797 Mar 22 18:18 efw-hotspot-2.9.57-1.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 1486082 Apr 8 11:29 efw-hotspot-2.9.58-1.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 1486143 Apr 9 15:34 efw-hotspot-2.9.59-1.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 1489902 Apr 16 14:33 efw-hotspot-2.9.61-1.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 1490882 Apr 17 09:14 efw-hotspot-2.9.63-1.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 1491034 Apr 18 11:53 efw-hotspot-2.9.64-1.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 834561 Mar 8 13:20 endian-artwork-enterprise-2.9.7-0.endian1.noarch.rpm
-rw-r--r-- 1 clamav 1000 834884 Mar 14 11:51 endian-artwork-enterprise-2.9.8-0.endian1.noarch.rpm
-rw-rw-r-- 1 clamav 1000 419 Mar 11 18:39 pkgs
drwxrwxr-x 2 clamav 1000 4096 Apr 18 17:54 repodata

r8-2:

root@stan:/var/www/repo/qa/2.5/r8/2 # ls -l
total 5272
-rw-rw-r-- 1 clamav 1000 49659 Apr 2 15:08 efw-backup-2.9.20-1.endian10.i586.rpm
-rw-rw-r-- 1 clamav 1000 71990 Mar 20 15:31 efw-base-2.9.15-1.endian27.noarch.rpm
-rw-rw-r-- 1 clamav 1000 72149 Apr 2 18:09 efw-base-2.9.16-1.endian27.noarch.rpm
-rw-rw-r-- 1 clamav 1000 3405 Mar 20 14:30 efw-demo-2.9.1-1.endian1.i586.rpm
-rw-rw-r-- 1 clamav 1000 3546 Mar 30 18:58 efw-demo-2.9.2-1.endian1.i586.rpm
-rw-rw-r-- 1 clamav 1000 57518 Apr 2 15:30 efw-endian-client-2.9.18-0.endian30.noarch.rpm
-rw-rw-r-- 1 clamav 1000 1487605 Apr 15 19:50 efw-hotspot-2.9.60-1.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 1491377 Apr 19 10:22 efw-hotspot-2.9.65-1.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 15124 Apr 2 16:40 efw-interfaceeditor-2.9.2-1.endian1.noarch.rpm
-rw-rw-r-- 1 clamav 1000 2865 Apr 2 16:40 efw-interfaceeditor-uplink-adsl-2.9.2-1.endian1.noarch.rpm
-rw-rw-r-- 1 clamav 1000 2660 Apr 2 16:40 efw-interfaceeditor-uplink-all-2.9.2-1.endian1.noarch.rpm
-rw-r--r-- 1 root root 2893 Apr 2 16:40 efw-interfaceeditor-uplink-analog-2.9.2-1.endian1.noarch.rpm
-rw-rw-r-- 1 clamav 1000 2870 Apr 2 16:41 efw-interfaceeditor-uplink-dhcp-2.9.2-1.endian1.noarch.rpm
-rw-rw-r-- 1 clamav 1000 2878 Apr 2 16:41 efw-interfaceeditor-uplink-gateway-2.9.2-1.endian1.noarch.rpm
-rw-rw-r-- 1 clamav 1000 2865 Apr 2 16:41 efw-interfaceeditor-uplink-isdn-2.9.2-1.endian1.noarch.rpm
-rw-rw-r-- 1 clamav 1000 2874 Apr 2 16:41 efw-interfaceeditor-uplink-pppoe-2.9.2-1.endian1.noarch.rpm
-rw-rw-r-- 1 clamav 1000 2860 Apr 2 16:41 efw-interfaceeditor-uplink-pptp-2.9.2-1.endian1.noarch.rpm
-rw-rw-r-- 1 clamav 1000 2884 Apr 2 16:42 efw-interfaceeditor-uplink-static-2.9.2-1.endian1.noarch.rpm
-rw-rw-r-- 1 clamav 1000 43304 Apr 11 23:42 efw-ipsec-2.9.5-1.endian7.i586.rpm
-rw-rw-r-- 1 clamav 1000 93789 Apr 12 11:41 efw-shell-2.9.7-0.endian2.noarch.rpm
-rw-r--r-- 1 clamav 1000 382795 Apr 12 19:40 efw-snort-2.9.2-1.endian20.noarch.rpm
-rw-rw-r-- 1 clamav 1000 61978 Apr 11 18:19 efw-vpn-2.9.8-0.endian14.noarch.rpm
-rw-rw-r-- 1 clamav 1000 54410 Apr 11 18:57 efw-vpnclient-2.9.7-0.endian15.noarch.rpm
-rw-rw-r-- 1 clamav 1000 486643 Apr 12 12:22 emi-2.9.15-0.endian9.noarch.rpm
-rw-rw-r-- 1 clamav 1000 309257 Apr 19 18:00 endian-core-2.9.11-0.endian11.i586.rpm
-rw-rw-r-- 1 clamav 1000 283613 Mar 19 20:33 jobsengine-2.9.22-1.endian4.i586.rpm
-rw-rw-r-- 1 clamav 1000 284365 Mar 30 15:56 jobsengine-2.9.23-1.endian4.i586.rpm
-rw-rw-r-- 1 clamav 1000 2635 May 6 18:16 pkgs
-rw------- 1 clamav 1000 19 Apr 12 19:19 pkgs.save
drwxr-xr-x 2 root root 4096 May 6 18:16 repodata

r8-3:

root@stan:/var/www/repo/qa/2.5/r8/3 # ls -l
total 31172
-rw-r--r-- 1 root root 49150 Mar 6 14:54 efw-backup-2.9.18-1.endian10.armv5tel.rpm
-rw-r--r-- 1 root root 49180 Mar 6 14:47 efw-backup-2.9.18-1.endian10.i586.rpm
-rw-r--r-- 1 root root 41903 Apr 24 18:20 efw-dnsmasq-2.9.15-0.endian18.armv5tel.rpm
-rw-r--r-- 1 root root 41909 Mar 26 11:14 efw-dnsmasq-2.9.15-0.endian18.i586.rpm
-rw-r--r-- 1 root root 57791 Apr 18 15:12 efw-endian-client-2.9.19-0.endian30.noarch.rpm
-rw-r--r-- 1 root root 57905 May 8 12:35 efw-endian-client-2.9.20-0.endian30.noarch.rpm
-rw-r--r-- 1 root root 43316 May 9 17:47 efw-ipsec-2.9.5-1.endian8.i586.rpm
-rw-r--r-- 1 root root 487043 Apr 30 12:23 emi-2.9.19-0.endian9.noarch.rpm
-rw-r--r-- 1 root root 137645 Apr 24 18:29 endian-client-2.9.18-1.endian28.armv5tel.rpm
-rw-r--r-- 1 root root 137669 Mar 29 19:00 endian-client-2.9.18-1.endian28.i586.rpm
-rw-r--r-- 1 root root 815005 May 7 18:41 freeradius-2.1.12-8.endian13.i586.rpm
-rw-r--r-- 1 root root 282245 Dec 6 08:58 jobsengine-2.9.19-1.endian4.i586.rpm
-rw-r--r-- 1 root root 29613631 May 2 16:35 ntop-4.1.0-4.endian8.i586.rpm
-rw-r--r-- 1 root root 1260 May 10 13:28 pkgs
-rw-r--r-- 1 root root 1444 May 10 13:24 pkgs+smartpm
drwxr-xr-x 2 root root 4096 May 10 13:28 repodata

After the fix:

root@upgrade_3:~ # smart upgrade --explain efw-endian-client
Loading cache...
Updating cache... ######################################################################################################## [100%]

Computing transaction...

Upgrading packages (1):
  efw-endian-client-4:2.9.20-0.endian30@noarch [custom-r8-3, r8-4]
    Upgrades:
      efw-endian-client-2:2.4.30-0.endian30@noarch (upgraded)
    Requires:
      jobsengine-2.9.23-1.endian4@i586 (installed)

Installing packages (2):
  jobsengine-2.9.23-1.endian4@i586 [r8-2, r8-4]
    Requires:
      python-setproctitle-1.1-0.endian1@i586 (installed)
    Required By:
      efw-endian-client-4:2.9.20-0.endian30@noarch (installed)
  python-setproctitle-1.1-0.endian1@i586 [r8-4]
    Required By:
      jobsengine-2.9.23-1.endian4@i586 (installed)

277.7kB of package files are needed. 1.3MB will be used.

Confirm changes? (Y/n):

which then installs, because jobsengine is part of the transaction