Merge pull request #18 from hseipp/master

Fix AWS deployment issues, add Object support for AWS

aws/README.md

@@ -9,15 +9,23 @@ After installing [SpectrumScaleVagrant and Vagrant itself](../README.md) you
 need to install the [vagrant-aws plugin](https://github.com/mitchellh/vagrant-aws)
 to enable Vagrant to manage virtual environments on AWS.
 
-On my host the installation of the vagrant-aws plugin failed, but [pinning the
+On Windows 10 hosts, the installation of the vagrant-aws plugin might fail, but [pinning the
 version of fog-ovirt](https://github.com/mitchellh/vagrant-aws/issues/539#issuecomment-398100794)
-resolved the issue:
+resolves the issue:
 
 ```
 vagrant plugin install --plugin-version 1.0.1 fog-ovirt
 vagrant plugin install vagrant-aws
 ```
 
+On Ubuntu 20.04 it is required to install an additional dependency to pass the vagrant-aws
+plugin installation:
+
+```
+sudo apt install libcurl4-openssl-dev
+vagrant plugin install vagrant-aws
+```
+
 ## Get the dummy box for the vagrant-aws plugin
 
 Vagrant requires each provider plug-in to provide its own box format. The
@@ -74,9 +82,9 @@ Copy the Spectrum Scale self-extracting installation package to `/software`, if
 SpectrumScaleVagrant\aws\prep-ami>vagrant ssh
 
 [centos@ip-172-31-27-143 ~]$ ls -l /software
-total 1489140
--rw-r--r--. 1 centos centos        136 Feb 16 17:26 README
--rw-rw-r--. 1 centos centos 1564495872 Feb 14 13:20 Spectrum_Scale_Data_Management-5.0.2.2-x86_64-Linux-install
+-rw-rw-r--.  1 centos centos        134 16. Jun 10:09 README
+-r-xr-xr-x.  1 centos centos 1407530361 16. Jun 13:11 Spectrum_Scale_Developer-5.1.1.0-x86_64-Linux-install
+-rw-rw-r--.  1 centos centos         88 16. Jun 13:11 Spectrum_Scale_Developer-5.1.1.0-x86_64-Linux-install.md5
 
 [centos@ip-172-31-27-143 ~]$ exit
 logout
@@ -85,7 +93,46 @@ Connection to ec2-34-224-86-55.compute-1.amazonaws.com closed.
 SpectrumScaleVagrant\aws\prep-ami>
 ```
 
-By default the official CentOS AMI and derived AMIs leave the orphaned root volume, after the owning virtual machine (instance) is terminated. Amazon charges for orphaned root volumes. They either need to be deleted manually, or the initial virtual machine needs to be modified, before the Spectrum Scale Base AMI is created. See the Amazon documentation ([Changing the Root Device Volume to Persist](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/RootDeviceStorage.html#Using_RootDeviceStorage)) for details. The procedure requires the [installation of the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html). Follow the AWS documentation for the installation of the AWS CLI.
+Having checked that `DeleteOnTermination` is set to `true` (see [Appendix](#appendix-avoid-orphaned-root-volumes)) we can build the Spectrum Scale AWS AMI and terminate the virtual machine:
+1. `vagrant package SpectrumScale_base --output SpectrumScale_base.box`
+1. `vagrant destroy`
+
+If `vagrant package` fails with the error message `Malformed => AMI names must be between 3 and 128 characters long, and may contain letters, numbers, '(', ')', '.', '-', '/' and '_'`
+you need to apply a patch described in the [Appendix](#appendix-fix-for-vagrant-aws-packaging) to your local copy of vagrant-aws and try again.
+
+
+## Configure the SpectrumScale_base AMI ID
+
+The `vagrant package ...` command of the previous step prints the AMI ID of the new SpectrumScale_base AMI:
+
+```
+...
+==> SpectrumScale_base: Waiting for the AMI 'ami-05d550f0ea6e84325' to burn...
+...
+```
+
+Copy the `Vagrantfile.aws-ami.sample` to `Vagrantfile.aws-ami` and update that file with the AMI ID of the SpectrumScale_base AMI:
+
+```
+cd SpectrumScaleVagrant\aws
+copy Vagrantfile.aws-ami.sample Vagrantfile.aws-ami
+notepad Vagrantfile.aws-ami
+```
+
+## Boot a virtual machine with a single node Spectrum Scale cluster
+
+Now we are ready to boot a virtual machine on AWS and to configure it with a single node Spectrum Scale cluster:
+1. `cd SpectrumScaleVagrant\aws`
+1. `vagrant up`
+1. `vagrant ssh`
+
+See the [README.md](../README.md) for details on the configured Spectrum Scale cluster.
+
+
+## Appendix: Avoid orphaned root volumes
+
+Versions of the official CentOS AMI and derived AMIs might leave the orphaned root volume, after the owning virtual machine (instance) is terminated.
+Amazon charges for orphaned root volumes. They either need to be deleted manually, or the initial virtual machine needs to be modified, before the Spectrum Scale Base AMI is created. See the Amazon documentation ([Changing the Root Device Volume to Persist](https://docs.aws.amazon.com/AWSEC2/latest/UserGuide/RootDeviceStorage.html#Using_RootDeviceStorage)) for details. The procedure requires the [installation of the AWS CLI](https://docs.aws.amazon.com/cli/latest/userguide/cli-chap-install.html). Follow the AWS documentation for the installation of the AWS CLI.
 
 First step is to query the `InstanceId` and the setting for `DeleteOnTermination` of the running virtual machine:
 
@@ -141,33 +188,26 @@ SpectrumScaleVagrant\aws\prep-ami>aws ec2 describe-instances --region us-east-1
 SpectrumScaleVagrant\aws\prep-ami>
 ```
 
-Having checked that `DeleteOnTermination` is set to `true` we can build the Spectrum Scale AWS AMI and terminate the virtual machine:
-1. `vagrant package SpectrumScale_base --output SpectrumScale_base.box`
-1. `vagrant destroy`
-
-## Configure the SpectrumScale_base AMI ID
-
-The `vagrant package ...` command of the previous step prints the AMI ID of the new SpectrumScale_base AMI:
-
-```
-...
-==> SpectrumScale_base: Waiting for the AMI 'ami-05d550f0ea6e84325' to burn...
-...
+## Appendix: Fix for vagrant-aws packaging
+
+If `vagrant package` fails with the error message `Malformed => AMI names must be between 3 and 128 characters long, and may contain letters, numbers, '(', ')', '.', '-', '/' and '_'`
+you need to apply a patch like the following to your local copy of vagrant-aws. You need to adopt to your installation path and user name:
+```
+--- /home/user/.vagrant.d/gems/2.7.0/gems/vagrant-aws-0.7.2/lib/vagrant-aws/action/package_instance.rb	2021-06-23 13:58:10.612642358 +0200
++++ /home/user/.vagrant.d/gems/2.7.0/gems/vagrant-aws-0.7.2/lib/vagrant-aws/action/package_instance_fixed.rb	2021-06-23 13:59:10.136684461 +0200
+@@ -39,11 +39,12 @@
+           begin
+             # Get the Fog server object for given machine
+             server = env[:aws_compute].servers.get(env[:machine].id)
++            servername = "#{server.tags["Name"]}".chomp!
+ 
+             env[:ui].info(I18n.t("vagrant_aws.packaging_instance", :instance_id => server.id))
+-            
++
+             # Make the request to AWS to create an AMI from machine's instance
+-            ami_response = server.service.create_image server.id, "#{server.tags["Name"]} Package - #{Time.now.strftime("%Y%m%d-%H%M%S")}", ""
++            ami_response = server.service.create_image server.id, "#{servername}-Package-#{Time.now.strftime("%Y%m%d-%H%M%S")}", ""
+ 
+             # Find ami id
+             @ami_id = ami_response.data[:body]["imageId"]
 ```
-
-Copy the `Vagrantfile.aws-ami.sample` to `Vagrantfile.aws-ami` and update that file with the AMI ID of the SpectrumScale_base AMI:
-
-```
-cd SpectrumScaleVagrant\aws
-copy Vagrantfile.aws-ami.sample Vagrantfile.aws-ami
-notepad Vagrantfile.aws-ami
-```
-
-## Boot a virtual machine with a single node Spectrum Scale cluster
-
-Now we are ready to boot a virtual machine on AWS and to configure it with a single node Spectrum Scale cluster:
-1. `cd SpectrumScaleVagrant\aws`
-1. `vagrant up`
-1. `vagrant ssh`
-
-See the [README.md](../README.md) for details on the configured Spectrum Scale cluster.

aws/Vagrantfile

@@ -29,6 +29,17 @@ load File.expand_path('../Vagrantfile.aws-ami', __FILE__)
 # Load common settings
 load File.expand_path('../../shared/Vagrantfile.common', __FILE__)
 
+# Workaround for vagrant-aws issue https://github.com/mitchellh/vagrant-aws/issues/566
+class Hash
+  def slice(*keep_keys)
+    h = {}
+    keep_keys.each { |key| h[key] = fetch(key) if has_key?(key) }
+    h
+  end unless Hash.method_defined?(:slice)
+  def except(*less_keys)
+    slice(*keys - less_keys)
+  end unless Hash.method_defined?(:except)
+end
 
 # Customize configuration specific settings
 Vagrant.configure("2") do |config|
@@ -52,11 +63,15 @@ Vagrant.configure("2") do |config|
     override.ssh.private_key_path = $my_aws_private_key_path
 
     # Pin the AWS region
+    # aws.region = "eu-central-1"
     aws.region = "us-east-1"
 
     # Pin the SpectrumScale_base AMI
     aws.ami = $my_aws_SpectrumScale_base_AMI_ID
 
+    # Set instance type - default is m3.medium - we need more RAM for CES
+    aws.instance_type = "m3.large"
+
     # Set user name for CentOS images
     override.ssh.username = "centos"
 
@@ -66,13 +81,13 @@ Vagrant.configure("2") do |config|
 
     # Add disks for SpectrumScale NSDs
     aws.block_device_mapping = [
-      { 'DeviceName' => '/dev/xvdb', 'Ebs.VolumeSize' =>  2 },
-      { 'DeviceName' => '/dev/xvdc', 'Ebs.VolumeSize' =>  2 },
-      { 'DeviceName' => '/dev/xvdd', 'Ebs.VolumeSize' =>  2 },
-      { 'DeviceName' => '/dev/xvde', 'Ebs.VolumeSize' =>  2 },
-      { 'DeviceName' => '/dev/xvdf', 'Ebs.VolumeSize' =>  2 },
-      { 'DeviceName' => '/dev/xvdg', 'Ebs.VolumeSize' => 10 },
-      { 'DeviceName' => '/dev/xvdh', 'Ebs.VolumeSize' => 10 },      
+      { 'DeviceName' => '/dev/xvdb', 'Ebs.VolumeSize' =>  3 },
+      { 'DeviceName' => '/dev/xvdc', 'Ebs.VolumeSize' =>  3 },
+      { 'DeviceName' => '/dev/xvdd', 'Ebs.VolumeSize' =>  3 },
+      { 'DeviceName' => '/dev/xvde', 'Ebs.VolumeSize' =>  3 },
+      { 'DeviceName' => '/dev/xvdf', 'Ebs.VolumeSize' =>  3 },
+      { 'DeviceName' => '/dev/xvdg', 'Ebs.VolumeSize' => 5 },
+      { 'DeviceName' => '/dev/xvdh', 'Ebs.VolumeSize' => 5 },
     ]
 
 

aws/prep-ami/Vagrantfile

@@ -14,7 +14,19 @@ EOT
 load File.expand_path('../../Vagrantfile.aws-credentials', __FILE__)
 
 # Load RPMs that need to be installed on top of the CentOS base image
-load File.expand_path('../../../shared/Vagrantfile.rpms', __FILE__)
+load File.expand_path('../../../shared/Vagrantfile8.rpms', __FILE__)
+
+# Workaround for vagrant-aws issue https://github.com/mitchellh/vagrant-aws/issues/566
+class Hash
+  def slice(*keep_keys)
+    h = {}
+    keep_keys.each { |key| h[key] = fetch(key) if has_key?(key) }
+    h
+  end unless Hash.method_defined?(:slice)
+  def except(*less_keys)
+    slice(*keys - less_keys)
+  end unless Hash.method_defined?(:except)
+end
 
 
 # Customize configuration specific settings
@@ -37,10 +49,14 @@ Vagrant.configure("2") do |config|
     override.ssh.private_key_path = $my_aws_private_key_path
 
     # Pin the AWS region
+    # aws.region = "eu-central-1"
     aws.region = "us-east-1"
 
-    # Pin the CentOS 7 AMI
-    aws.ami = "ami-02eac2c0129f6376b"
+    # Pin the CentOS 8 AMI
+    # CentOS 8 AMI for eu-central-1
+    # aws.ami = "ami-04c21037b3f953d37"
+    # CentOS 8 AMI for us-east-1
+    aws.ami = "ami-0d6e9a57f6259ba3a"
 
     # Set user name for CentOS images
     override.ssh.username = "centos"

setup/demo/script-08.sh

@@ -1,5 +1,12 @@
 #!/usr/bin/bash
 
+usage(){
+  echo "Usage: $0 [<provider>]"
+  echo "Supported provider:"
+  echo "  AWS"
+  echo "  VirtualBox"
+  echo "  libvirt"
+}
 
 # Improve readability of output
 echo "========================================================================================="
@@ -15,15 +22,37 @@ set -x
 # Exit script immediately, if one of the commands returns error code
 set -e
 
+# Exit, if not exactly one argument given
+if [ $# -ne 1 ]; then
+  usage
+  exit -1
+fi
+
+# Use first argument as current underlying provider
+case $1 in
+  'AWS'|'VirtualBox'|'libvirt' )
+    PROVIDER=$1
+    ;;
+  *)
+    usage
+    exit -1
+    ;;
+esac
+
 # Prepare Object Storage to accept AWSv4 Authentication
 echo "===> Preparing S3 Object for AWSv4 Authentication - use US as region with your client software"
 sudo mmobj config change --ccrfile "proxy-server.conf" --section "filter:s3api" --property "location" --value "US"
 
 # Copy credentials file to user directory and use it
 echo "===> Copy object credentials file and source it"
-sudo cp /root/openrc /home/vagrant/openrc
-sudo chown vagrant:vagrant /home/vagrant/openrc
-source /home/vagrant/openrc
+TARGET=vagrant
+if [ "$PROVIDER" = "AWS" ]
+then
+    TARGET=centos
+fi
+sudo cp /root/openrc /home/$TARGET/openrc
+sudo chown $TARGET:$TARGET /home/$TARGET/openrc
+source /home/$TARGET/openrc
 
 # Create S3 test user with access to the existing admin project
 echo "===> Create S3 tests user within admin project"

setup/demo/script.sh

@@ -47,10 +47,7 @@ esac
 /vagrant/demo/script-05.sh
 /vagrant/demo/script-06.sh
 #/vagrant/demo/script-07.sh
-if [ "$PROVIDER" = "VirtualBox" -o "$PROVIDER" = "libvirt" ]
-then
-    /vagrant/demo/script-08.sh
-fi
+/vagrant/demo/script-08.sh $PROVIDER
 
 # Tweak the configuration to show more management capabilities
 /vagrant/demo/script-80.sh

setup/install/script-04.sh

@@ -63,7 +63,8 @@ sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale node list
 # ... for AWS
 if [ "$PROVIDER" = "AWS" ]
 then
-  sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale nsd add -p m1.example.com -fs fs1 /dev/xvdb /dev/xvdc /dev/xvdd /dev/xvde /dev/xvdf
+  sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale nsd add -p m1.example.com -fs fs1 /dev/xvdb /dev/xvdc /dev/xvdd
+  sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale nsd add -p m1.example.com -fs cesShared /dev/xvde /dev/xvdf
   sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale nsd add -p m1.example.com /dev/xvdg /dev/xvdh
 fi
 # ... for VirtualBox

setup/install/script-08.sh

@@ -8,9 +8,6 @@ usage(){
   echo "  libvirt"
 }
 
-# Exit on error
-set -e
-
 # Improve readability of output
 echo "========================================================================================="
 echo "===>"
@@ -49,7 +46,8 @@ sudo dnf -y upgrade
 
 echo "===> configuring Object"
 sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale node add m1.example.com -p
-sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale config protocols -e 192.168.56.101
+CESVIP=$(grep "cesip" /etc/hosts | awk {'print $1'})
+sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale config protocols -e $CESVIP
 sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale config protocols -f cesShared -m /ibm/cesShared
 sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale enable object
 sudo /usr/lpp/mmfs/5.1.1.0/ansible-toolkit/spectrumscale config object -f fs1 -m /ibm/fs1 -e cesip.example.com -au admin -dp passw0rd -sp passw0rd -ap passw0rd

setup/install/script.sh

@@ -51,11 +51,7 @@ esac
 /vagrant/install/script-05.sh $PROVIDER
 /vagrant/install/script-06.sh $PROVIDER
 /vagrant/install/script-07.sh $PROVIDER
-# Do not install Object on AWS (yet)
-if [ "$PROVIDER" = "VirtualBox" -o "$PROVIDER" = "libvirt" ]
-then
-  /vagrant/install/script-08.sh $PROVIDER
-fi
+/vagrant/install/script-08.sh $PROVIDER
 # Exit successfully
 echo "===> Script completed successfully!"
 exit 0

shared/Vagrantfile.common

@@ -26,6 +26,18 @@ Vagrant.configure('2') do |config|
       name:   "Configure /etc/hosts for AWS",
       inline: "
         echo \"`hostname -I` m1.example.com m1\" >> /etc/hosts
+        # Get primary IP
+        IPA=$(hostname -I | awk '{print $1}')
+        # Extract IP Range
+        IPRANGE=${IPA%.*}
+        # Try .200 as VIP
+        NEWIP=${IPRANGE}.200
+        if [ \"$NEWIP\" == \"$IPA\" ];
+        then
+          # In case 200 is taken, use different one
+          NEWIP=${IPRANGE}.201
+        fi
+        echo \"$NEWIP cesip.example.com cesip\" >> /etc/hosts
       "
   end
   # ... for VirtualBox
@@ -56,11 +68,14 @@ Vagrant.configure('2') do |config|
    "
 
   # Get fingerprint for management IP address
-  config.vm.provision "shell",
-    name:   "Get fingerprint for management IP address",
-    inline: "
-      ssh-keyscan -t ecdsa m1m.example.com >> /root/.ssh/known_hosts 2>/dev/null
-   "
+  if $SpectrumScaleVagrant_provider == 'VirtualBox' || $SpectrumScaleVagrant_provider == 'libvirt'
+  then
+    config.vm.provision "shell",
+      name:   "Get fingerprint for management IP address",
+      inline: "
+        /usr/bin/sudo /usr/bin/bash -c '/usr/bin/ssh-keyscan -t ecdsa m1m.example.com >> /root/.ssh/known_hosts'
+     "
+  end
 
   # Add Spectrum Scale executables to $PATH
   config.vm.provision "shell",