Pacemaker service

chef/cookbooks/pacemaker/README.md

@@ -1,61 +1,82 @@
 DESCRIPTION
 ===========
-
-Installs pacemaker and corosync.
-
-This cookbook handles the creation of a pacemaker cluster with Corosync as the
-transfer layer.
+This cookbook handles the creation of a Pacemaker cluster with Corosync as the transfer layer.
 
 A pacemaker cluster consists of one master node, and one or more client nodes.
 
-HA services have a VIP (Virtual IP address) that "floats" between nodes. This address
-will only be active on one node at a time, and is managed by pacemaker.
+HA services have a VIP (Virtual IP address) that "floats" between nodes. This address will only be active on one node at a time, and is managed by pacemaker.
 
 In order for this to work, we need to set ip_nonlocal_bind on each node:
 
 echo 1 > /proc/sys/net/ipv4/ip_nonlocal_bind
 
-Also, the services being managed (haproxy, httpd, whatever) will be managed by
-pacemaker. So, we need to be able to "trick" the other recipes configuring those
-services into NOT actually starting/maintaining the services in a running state.
-They will be started only on the active node, so the running status of the services
-will float along with the VIP
-
-What we expect to be able to do:
-
-# Install a pacemaker cluster
-## Create the corosync cert on the master node
-## Store the cert for use by the clients
-# Set up a VIP for the HA services (set via attribute)
-# Manage HA services by pacemaker.
-## Services will be listed as attributes in a hash
-## Service name will be the key of the hash
-## Relevant data required will be values under the service name
+Also, the services being managed (haproxy, httpd, whatever) will be managed by pacemaker. So, we need to be able to "trick" the other recipes configuring those services into NOT actually starting/maintaining the services in a running state. They will be started only on the active node, so the running status of the services will float along with the VIP
 
 Example:
 
 default[pacemaker][services] => {
 	"mysqld" => {
-		"datadir" => "/var/lib/mysql"
+    "vip" => "10.0.111.5",
+    "active" => "10.0.111.2",
+    "passive" => ["10.0.111.3"]
 	},
 	"apache2" => {}
-
 }
 
 
-Currently, the only configurable attribute is the master node. The value for this attribute MUST
-be the FQDN of the node designated as the pacemaker cluster master. We store configuration data
-as node attributes (generated at the time of convergence) on this node, and the other
-cluster members read this via chef search. So get the FQDN right!
+Currently, the only configurable attribute is the master node. The value for this attribute MUST be the FQDN of the node designated as the pacemaker cluster master. We store configuration data as node attributes (generated at the time of convergence) on this node, and the other cluster members read this via chef search.
+
+Requirements
+============
+It is highly recommended you use the `ntp` cookbook to keep your machines in sync.
 
+Resource/Provider
+=================
+pacemaker_service
+---------------
+LWRP for managing services with Pacemaker. Pacemaker will control the services for the node transparently, while other recipes will continue to believe they are managing that service. This is done by removing the init script and symlinking it to /bin/true while Pacemaker actually handles the script (this needs to be sorted out).
+
+# Actions
+- :create: Have Pacemaker manage the service.
+- :remove: Have the service to manage itself.
+
+# Attribute Parameters
+- service: name of the service to manage with pacemaker. Name attribute.
+- vip: virtual IP address to map to the service.
+- active: whether this node is the 'active' node. Defaults to `false` for the 'passive' node.
+- path: path to init script if it does not path to '/etc/init.d/SERVICE'. (optional)
+
+# Examples
+    # add the mysqld service
+    pacemaker_service "mysqld" do
+      vip "10.0.111.5"
+      active true
+      action :create
+    end
+
+DONE
+====
+# Install a pacemaker cluster
+## Create the corosync cert on the master node
+## Store the cert for use by the clients
+
+TODO
+====
+# Set up a VIP for the HA services (set via attribute)
+# Manage HA services by pacemaker.
+## Services will be listed as attributes in a hash
+## Service name will be the key of the hash
+## Relevant data required will be values under the service name
 
 
 LICENSE AND AUTHOR
 ==================
 
 Author:: Keith Hudgins (<[email protected]>)
+Author:: Matt Ray (<[email protected]>)
 
 Copyright 2011, Dell, Inc.
+Copyright 2011, Opscode, Inc.
 
 Licensed under the Apache License, Version 2.0 (the "License");
 you may not use this file except in compliance with the License.

chef/cookbooks/pacemaker/attributes/default.rb

@@ -1 +1 @@
-default['pacemaker']['master'] = 'd00-0c-29-33-0e-c4.greenman.org'
+default['pacemaker']['services'] = {}

chef/cookbooks/pacemaker/examples/roles/pacemaker-active.rb

@@ -0,0 +1,22 @@
+name "pacemaker-active"
+description "pacemaker active."
+
+override_attributes(
+  "pacemaker" => {
+    "services" => {
+      "apache2" => {
+        "vip" => "10.0.111.5",
+        "active" => "ubuntu1-1004.vm"
+      },
+      "mysql" => {
+        "vip" => "10.0.111.7",
+        "passive" => ["ubuntu1-1004.vm"]
+      }
+    }
+  }
+  )
+
+run_list(
+  "recipe[pacemaker::master]",
+  "recipe[pacemaker::services]"
+  )

chef/cookbooks/pacemaker/examples/roles/pacemaker.rb

@@ -0,0 +1,14 @@
+name "pacemaker-passive"
+description "pacemaker passive."
+
+override_attributes(
+  "pacemaker" => {
+    "services" => {
+    }
+  }
+  )
+
+run_list(
+  "recipe[pacemaker::client]",
+  "recipe[pacemaker::services]"
+  )

chef/cookbooks/pacemaker/files/default/corosync

@@ -0,0 +1,3 @@
+# managed by Chef
+# start corosync at boot [yes|no]
+START=yes

chef/cookbooks/pacemaker/metadata.rb

@@ -2,5 +2,5 @@
 maintainer_email "[email protected]"
 license          "Apache 2.0"
 description      "Installs/Configures Pacemaker/Corosync"
-long_description IO.read(File.join(File.dirname(__FILE__), 'README.me'))
+long_description IO.read(File.join(File.dirname(__FILE__), 'README.md'))
 version          "0.1"

chef/cookbooks/pacemaker/providers/service.rb

@@ -0,0 +1,68 @@
+#
+# Author:: Matt Ray (<[email protected]>)
+# Cookbook Name:: pacemaker
+# Provider:: service
+#
+# Copyright:: 2011, Opscode, Inc
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+# actions :create, :remove
+
+# attribute :service, :kind_of => String, :name_attribute => true
+# attribute :vip, :kind_of => String
+# attribute :active, :default => false
+# attribute :path, :kind_of => String
+
+action :create do
+  service = new_resource.service
+  vip = new_resource.vip
+  active = new_resource.active
+  path = new_resource.path
+  Chef::Log.info "pacemaker_service #{service} #{vip} #{active} #{path}"
+  oldservice = node['pacemaker']['services'][service]
+  newservice = {}
+  newservice['vip'] = vip
+  if active
+    newservice['active'] = node.name
+  else
+    #search for active?
+    #newservice['active'] =
+    if oldservice
+      newservice['active'] = oldservice['active']
+      newservice['passive'] = oldservice['passive']
+      newservice['passive'].push(node.name)
+      newservice['passive'].uniq!.sort!
+    else
+      newservice['passive'] = [node.name]
+    end
+  end
+  #compare with previous state
+  if newservice != oldservice
+    #put the service into the attributes of the node
+    node['pacemaker']['services'][service] = newservice
+    #figure out how pacemaker handles services
+    new_resource.updated_by_last_action(true)
+  end
+end
+
+action :remove do
+  service = new_resource.service
+  if node['pacemaker']['services'][service]
+    #remove the parameters into the attributes of the node
+    node['pacemaker']['services'].delete(service)
+    #figure out how to restore services from pacemaker control
+    new_resource.updated_by_last_action(true)
+  end
+end

chef/cookbooks/pacemaker/recipes/client.rb

@@ -1,9 +1,29 @@
-include_recipe "pacemaker::default"
 require 'base64'
 
-# Don't need haveged, we're not generating certs
+# Install haveged to create entropy so keygen doesn't take an hour
+%w{ corosync pacemaker }.each do |pkg|
+  package pkg
+end
+
+authkey = ""
 
 # Find the master node:
-master = search(:node, "fqdn:#{node['pacemaker']['master']}")
+if !File.exists?("/etc/corosync/authkey")
+  master = search(:node, "corosync:authkey")
+  if master.nil? or (master.length > 1)
+    Chef::Application.fatal! "You must have one node with the pacemaker::master recipe in their run list to be a client."
+  end
+  Chef::Log.info "Found pacemaker::master node: #{master[0].name}"
+  authkey = Base64.decode64(master[0]['corosync']['authkey'])
+end
 
-log "Master node is #{master['ip_address']}"
+file "/etc/corosync/authkey" do
+  not_if {File.exists?("/etc/corosync/authkey")}
+  content authkey
+  owner "root"
+  mode "0400"
+  action :create
+end
+
+#manage the corosync services
+include_recipe "pacemaker::default"

chef/cookbooks/pacemaker/recipes/default.rb

@@ -13,16 +13,38 @@
 # limitations under the License.
 #
 
+# template "/etc/ha.cf" do
+#   source 'ha.cf.erb'
+#   owner 'root'
+#   group 'root'
+#   mode '0644'
+# end
 
-%w{ pacemaker corosync }.each do |pkg|
-  package pkg
+#enable the corosync service
+cookbook_file "/etc/default/corosync" do
+  source "corosync"
+  owner "root"
+  group "root"
+  mode "0644"
 end
 
-template "/etc/ha.cf" do
-  source 'ha.cf.erb'
-  owner 'root'
-  group 'root'
-  mode '0644'
+#get the first 3 quads of the IP and add '0'
+bindnetaddr = node.ipaddress[0..node.ipaddress.rindex('.')]+'0'
+
+template "/etc/corosync/corosync.conf" do
+  source "corosync.conf.erb"
+  owner "root"
+  group "root"
+  mode "0644"
+  variables :bindnetaddr => bindnetaddr
 end
 
+#start up the corosync service
+service "corosync" do
+  supports :restart => true, :status => :true
+  action [:enable, :start]
+  subscribes :restart, resources(:template => "/etc/corosync/corosync.conf"), :immediately
+end
 
+#disable stonith
+#crm configure property stonith-enabled=false

chef/cookbooks/pacemaker/recipes/master.rb

@@ -1,24 +1,26 @@
-include_recipe "pacemaker::default"
 require 'base64'
 
 # Install haveged to create entropy so keygen doesn't take an hour
+# odd errors coming out of automated installation, gets restarted next
+package "haveged" do
+  ignore_failure true
+end
 
-package "haveged"
-
-# Make sure haveged is running (Should already be on ubuntu)
+%w{ corosync pacemaker }.each do |pkg|
+  package pkg
+end
 
 service "haveged" do
   supports :restart => true, :status => :true
   action [:enable, :start]
 end
-
 
-execute "Create authkey" do
-  command "corosync-keygen"
+#create authkey
+execute "corosync-keygen" do
   creates "/etc/corosync/authkey"
-  action :run
   user "root"
   umask "0400"
+  action :run
 end
 
 # Read authkey (it's binary) into encoded format and save to chef server
@@ -33,6 +35,9 @@
     node.set_unless['corosync']['authkey'] = packed
     node.save
   end
+  action :nothing
+  subscribes :create, resources(:execute => "corosync-keygen"), :immediately
 end
 
-
+#manage the corosync services
+include_recipe "pacemaker::default"

chef/cookbooks/pacemaker/recipes/services.rb

@@ -0,0 +1,34 @@
+#
+# Author:: Matt Ray (<[email protected]>)
+# Cookbook Name:: pacemaker
+# Recipe:: services
+#
+# Copyright 2011, Opscode, Inc.
+#
+# Licensed under the Apache License, Version 2.0 (the "License");
+# you may not use this file except in compliance with the License.
+# You may obtain a copy of the License at
+#
+#     http://www.apache.org/licenses/LICENSE-2.0
+#
+# Unless required by applicable law or agreed to in writing, software
+# distributed under the License is distributed on an "AS IS" BASIS,
+# WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+# See the License for the specific language governing permissions and
+# limitations under the License.
+#
+
+node['pacemaker']['services'].keys.each do |svc|
+  Chef::Log.debug "Pacemaker::services #{svc}"
+  Chef::Log.debug node['pacemaker']['services'][svc]
+  isactive = false
+  if node['pacemaker']['services'][svc]['active'].eql?(node.name)
+    isactive = true
+  end
+  pacemaker_service svc do
+    vip node['pacemaker']['services'][svc]['vip']
+    active isactive
+    path node['pacemaker']['services'][svc]['path']
+    action :create
+  end
+end