debian-7.11 image fails under test kitchen (libc issues)

[briancline]

After generating a fresh cookbook with chef generate cookbook today, making some recipe changes, and setting the test kitchen platforms to debian-7.11, it fails to converge because the Chef Omnibus deb package it downloads contains a ruby that was compiled against libc 2.14. The current bento/debian-7.11 image contains libc 2.13. This was after a fresh download of the box image.

It's pretty clear that Omnitruck is recommending this deb for this platform/version pair, but not 100% sure whether this would be a Bento thing or a Omnibus package thing, so starting here.

Libc package versions on the VM:

ii  libc-bin                           2.13-38+deb7u11                amd64        Embedded GNU C Library: Binaries
ii  libc-dev-bin                       2.13-38+deb7u11                amd64        Embedded GNU C Library: Development binaries
ii  libc6:amd64                        2.13-38+deb7u11                amd64        Embedded GNU C Library: Shared libraries
ii  libc6-dev:amd64                    2.13-38+deb7u11                amd64        Embedded GNU C Library: Development Libraries and Header Files

Converge output:

% kitchen converge
-----> Starting Kitchen (v1.15.0)
-----> Creating <default-debian-711>...
       Bringing machine 'default' up with 'virtualbox' provider...
       ==> default: Importing base box 'bento/debian-7.11'...
==> default: Matching MAC address for NAT networking...
       ==> default: Checking if box 'bento/debian-7.11' is up to date...
       ==> default: Setting the name of the VM: kitchen-leloo_multipass-default-debian-711_default_1490738527173_65629
       ==> default: Fixed port collision for 22 => 2222. Now on port 2201.
       ==> default: Clearing any previously set network interfaces...
       ==> default: Preparing network interfaces based on configuration...
           default: Adapter 1: nat
           default: Adapter 2: hostonly
       ==> default: Forwarding ports...
           default: 22 (guest) => 2201 (host) (adapter 1)
       ==> default: Booting VM...
       ==> default: Waiting for machine to boot. This may take a few minutes...
           default: SSH address: 127.0.0.1:2201
           default: SSH username: vagrant
           default: SSH auth method: private key
           default:
           default: Vagrant insecure key detected. Vagrant will automatically replace
           default: this with a newly generated keypair for better security.
           default:
           default: Inserting generated public key within guest...
           default: Removing insecure key from the guest if it's present...
           default: Key inserted! Disconnecting and reconnecting using new SSH key...
       ==> default: Machine booted and ready!
       ==> default: Checking for guest additions in VM...
       ==> default: Setting hostname...
       ==> default: Configuring and enabling network interfaces...
       ==> default: Mounting shared folders...
           default: /tmp/omnibus/cache => /Users/bc/.kitchen/cache
       ==> default: Machine not provisioned because `--no-provision` is specified.
       [SSH] Established
       Vagrant instance <default-debian-711> created.
       Finished creating <default-debian-711> (0m31.94s).
-----> Converging <default-debian-711>...
       Preparing files for transfer
       Preparing dna.json
       Resolving cookbook dependencies with Berkshelf 5.6.0...
       Removing non-cookbook files before transfer
       Preparing validation.pem
       Preparing client.rb
-----> Installing Chef Omnibus (install only if missing)
       Downloading https://omnitruck.chef.io/install.sh to file /tmp/install.sh
       Trying wget...
       Download complete.
       debian 7 x86_64
       Getting information for chef stable  for debian...
       downloading https://omnitruck.chef.io/stable/chef/metadata?v=&p=debian&pv=7&m=x86_64
         to file /tmp/install.sh.2944/metadata.txt
       trying wget...
       sha1 f44f39d8059c93dbcb5d37f57111aad3a6c026cb
       sha256   292651ac21e093a40446da6b9a9b075ad31be6991a6f7ab63d5b6c2edabaa03c
       url  https://packages.chef.io/files/stable/chef/12.19.36/debian/7/chef_12.19.36-1_amd64.deb
       version  12.19.36
       downloaded metadata file looks valid...
       /tmp/omnibus/cache/chef_12.19.36-1_amd64.deb already exists, verifiying checksum...
       Comparing checksum with sha256sum...
       checksum compare succeeded, using existing file!

       WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING

       You are installing an omnibus package without a version pin.  If you are installing
       on production servers via an automated process this is DANGEROUS and you will
       be upgraded without warning on new releases, even to new major releases.
       Letting the version float is only appropriate in desktop, test, development or
       CI/CD environments.

       WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING WARNING

       Installing chef
       installing with dpkg...
       Selecting previously unselected package chef.
(Reading database ... 27471 files and directories currently installed.)
       Unpacking chef (from .../chef_12.19.36-1_amd64.deb) ...
       Setting up chef (12.19.36-1) ...
       Thank you for installing Chef!
       Transferring files to <default-debian-711>
       /opt/chef/embedded/bin/ruby: /lib/x86_64-linux-gnu/libc.so.6: version `GLIBC_2.14' not found (required by /opt/chef/embedded/lib/libruby.so.2.3)
>>>>>> ------Exception-------
>>>>>> Class: Kitchen::ActionFailed
>>>>>> Message: 1 actions failed.
>>>>>>     Converge failed on instance <default-debian-711>.  Please see .kitchen/logs/default-debian-711.log for more details
>>>>>> ----------------------
>>>>>> Please see .kitchen/logs/kitchen.log for more details
>>>>>> Also try running `kitchen diagnose --all` for configuration

My kitchen.yml file:

---
driver:
  name: vagrant

provisioner:
  name: chef_zero
  always_update_cookbooks: true

verifier:
  name: inspec

platforms:
  - name: debian-7.11

suites:
  - name: default
    driver:
      vm_hostname: dev-leloomultipass
      network:
        - ['private_network', {ip: '25.25.2.2'}]
    run_list:
      - recipe[leloo_multipass::default]
    verifier:
      inspec_tests:
        - test/smoke/default
    attributes:

[cheeseplus]

This is related to test-kitchen/kitchen-vagrant#293

The issue comes from when we cache this binary chef_12.19.36-1_amd64.deb, it is exactly the same name as the binary for other Ubuntu's so the caching uses it instead. Disabling caching or nuking the build in question from the cache is the workaround for the time being.

[cheeseplus]

Going to close here as this has been linked to the real upstream issue and we've got a workaround. If you want to chime in on the parent issue feel free but we're currently pursuing a fix.

[briancline]

Thank you!