Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[WIP] Complete rework of the role to simplify distributed site setup #53

Open
wants to merge 34 commits into
base: master
Choose a base branch
from
+928 −633
Open

[WIP] Complete rework of the role to simplify distributed site setup #53

Show file tree
Hide file tree
Changes from 20 commits
Commits
Show all changes
34 commits
Select commit Hold shift + click to select a range
5075df2
Remove unneeded variables
ganto Feb 13, 2017
66259aa
Add current status of site creation
ganto Mar 29, 2017
ee665af
Fix 'multisite_password', set required 'multisite_alias'
ganto Mar 30, 2017
7b8dbc2
Set credential paths related to the inventory name instead of fqdn
ganto Mar 30, 2017
821688e
Convert user setup to run for distributed sites in 'checkmk_servers__…
ganto Mar 30, 2017
21a2bfa
Iterate user setup tasks over all distributed sites
ganto Mar 30, 2017
2954cc2
Make distributed site login and config work with new site definition
ganto Mar 31, 2017
231d06e
Remove unused variables
ganto Apr 3, 2017
9128633
Create new env sub-role for site setup, write site facts
ganto Apr 10, 2017
3ea7723
Adjust home that it matches the user created by 'omd'
ganto Apr 10, 2017
333b001
Use user__dependent_accounts and etc_services__dependent_list from facts
ganto Apr 10, 2017
6b8fd58
Make sure the checkmk_server/env tasks are run by all play hosts
ganto Apr 11, 2017
1737762
Leverage ssh key setup from debops.users role, set public key local fact
ganto Apr 11, 2017
81f078a
Read correct facts after changes, add SSH authorized_keys setup
ganto Apr 12, 2017
a726abf
Add users to 'sshusers' group which allows SSH login
ganto Apr 12, 2017
81810fd
Run config synchronization to slave site via rsync
ganto Apr 12, 2017
e51f79f
Update playbook to use the 'checkmk_server/env' and 'debops.users' roles
ganto Apr 13, 2017
0d0a2fb
Rename 'checkmk_server__hostname' to 'checkmk_server__fqdn' for consi…
ganto Apr 20, 2017
5b01f6c
Fix applying custom patches, remove env role debugging
ganto Apr 20, 2017
bae1222
Leverage 'debops.apache' role for local reverse proxy configuration
ganto Apr 21, 2017
53c1e53
Explicitly define Apache configuration includes provided by Check_MK
ganto Apr 26, 2017
97aceb8
Remove HTTP-related ferm rules now handled by 'apache__ferm__dependen…
ganto Apr 21, 2017
1924a7a
Generate ferm rules for livestatus according to the configured sites
ganto May 2, 2017
5635218
Fix 'apache__dependent_snippets' configuration
ganto May 4, 2017
b2c4951
Remove some debugging code
ganto May 5, 2017
af9d305
Define dependent vars in defaults, don't fail with no facts
ganto May 5, 2017
cdfb4e3
Run tests on Ubuntu trusty as precise is EOL
ganto May 5, 2017
c5717d4
Create master site if 'checkmk_server__site' is defined
ganto May 5, 2017
35bc317
Don't fail if 'mod_headers' is not (yet) enabled
ganto May 8, 2017
e641ae7
Properly set and update local facts from the correct context
ganto May 15, 2017
5bc3e41
Adjust the fact that local facts are stored as dict not lists
ganto May 16, 2017
5c0a5da
Adjust SSH public key lookup to new facts layout
ganto May 16, 2017
6a5c4f5
Update dependency to Ansible >=2.3.0
ganto May 18, 2017
4d0e8ca
Clarify comments in the local facts template
ganto May 19, 2017
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
172 changes: 45 additions & 127 deletions defaults/main.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,10 +19,11 @@
checkmk_server__version: '1.2.8p21'

# ]]]
# .. envvar:: checkmk_server__version_label [[[
# .. envvar:: checkmk_server__version_suffix [[[
#
# Check_MK version label used with the :command:`omd` tool.
checkmk_server__version_label: '{{ checkmk_server__version }}.cre'
# Check_MK version suffix to :envvar:`checkmk_server__version` used with the
# :command:`omd` tool.
checkmk_server__version_suffix: '.cre'

# ]]]
# .. envvar:: checkmk_server__site_update [[[
Expand All @@ -37,9 +38,9 @@ checkmk_server__site_update: False
# Custom patches to apply after installing Check_MK package
checkmk_server__patches:
- patch: 'check-mk-raw-1.2.8-set-https-proxy-header.patch'
file: '/omd/versions/{{ checkmk_server__version_label }}/skel/etc/apache/apache-own.conf'
file: '/omd/versions/{{ checkmk_server__version }}{{ checkmk_server__version_suffix }}/skel/etc/apache/apache-own.conf'
- patch: 'check-mk-raw-1.2.8p4-read-X-Forwarded-Port-header.patch'
file: '/omd/versions/{{ checkmk_server__version_label }}/skel/etc/apache/conf.d/omd.conf'
file: '/omd/versions/{{ checkmk_server__version }}{{ checkmk_server__version_suffix }}/skel/etc/apache/conf.d/omd.conf'

# ]]]
# .. envvar:: checkmk_server__ferm_dependent_rules [[[
Expand All @@ -50,7 +51,6 @@ checkmk_server__ferm_dependent_rules: '{{
(checkmk_server__ferm_livestatus_rules if checkmk_server__multisite_livestatus else [])
}}'
# ]]]

# .. envvar:: checkmk_server__ferm_web_rules [[[
#
# Firewall configuration for WATO Web access.
Expand Down Expand Up @@ -88,16 +88,6 @@ checkmk_server__web_allow: []
# Check_MK Livestatus TCP socket. If list is empty, anyone can connect.
checkmk_server__livestatus_allow: []

# ]]]
# .. envvar:: checkmk_server__etc_services__dependent_list [[[
#
# Add entry for Livestatus to :file:`/etc/services` using the
# ``debops.etc_services`` role.
checkmk_server__etc_services__dependent_list:
- name: 'check-mk-livestatus'
port: '{{ checkmk_server__livestatus_port }}'
comment: 'Check_MK server Livestatus'

# ]]]
# .. envvar:: checkmk_server__livestatus_port [[[
#
Expand Down Expand Up @@ -138,22 +128,20 @@ checkmk_server__prerequisite_packages: [ 'apache2', 'python-passlib' ]
checkmk_server__site: 'debops'

# ]]]
# .. envvar:: checkmk_server__hostname [[[
# .. envvar:: checkmk_server__fqdn [[[
#
# Set Check_MK server DNS hostname (e. g. for agent download, API calls, ...).
# FIXME: Rename to checkmk_server__fqdn.
checkmk_server__hostname: '{{ ansible_local.core.fqdn
if (ansible_local|d() and ansible_local.core|d() and
ansible_local.core.fqdn|d())
else ansible_fqdn }}'
checkmk_server__fqdn: '{{ ansible_local.core.fqdn
if (ansible_local|d() and ansible_local.core|d() and
ansible_local.core.fqdn|d())
else ansible_fqdn }}'

# ]]]
# .. envvar:: checkmk_server__site_url [[[
#
# Check_MK server site URL.
checkmk_server__site_url: '{{ ("https://" if checkmk_server__pki else "http://") +
checkmk_server__hostname + "/" +
checkmk_server__site
checkmk_server__fqdn + "/" + checkmk_server__site
if checkmk_server__site|d() else "" }}'

# ]]]
Expand All @@ -171,11 +159,10 @@ checkmk_server__webapi_url: '{{ checkmk_server__site_url + "/check_mk/webapi.py"
# :ref:`checkmk_server__ref_omd_config` for more details.
checkmk_server__omd_config: '{{
checkmk_server__omd_config_email +
checkmk_server__omd_config_core +
(checkmk_server__omd_config_livestatus if checkmk_server__multisite_livestatus|d() else [])
checkmk_server__omd_config_core
}}'
# ]]]

# ]]]
# .. envvar:: checkmk_server__omd_config_email [[[
#
# Administrator email address set via OMD.
Expand All @@ -191,16 +178,6 @@ checkmk_server__omd_config_core:
- var: 'CORE'
value: 'icinga'

# ]]]
# .. envvar:: checkmk_server__omd_config_livestatus [[[
#
# Livestatus service configuration via OMD.
checkmk_server__omd_config_livestatus:
- var: 'LIVESTATUS_TCP'
value: 'on'
- var: 'LIVESTATUS_TCP_PORT'
value: '{{ checkmk_server__livestatus_port }}'

# ]]]
# .. envvar:: checkmk_server__sshkeys [[[
#
Expand Down Expand Up @@ -235,20 +212,6 @@ checkmk_server__ssh_arguments: '-o BatchMode=yes -o StrictHostKeyChecking=no -o
# Multisite Web Configuration [[[
# -------------------------------

# .. envvar:: checkmk_server__multisite_slave [[[
#
# Indicate if this site is a distributed monitoring slave which receives the
# Check_MK configuration from another Check_MK server instance.
checkmk_server__multisite_slave: False

# ]]]
# .. envvar:: checkmk_server__multisite_livestatus [[[
#
# Enable multisite Livestatus service. This is required for distributed
# monitoring of this site.
checkmk_server__multisite_livestatus: '{{ True if checkmk_server__multisite_slave|d() else False }}'

# ]]]
# .. envvar:: checkmk_server__multisite_config_path [[[
#
# Configuration path for Check_MK multisite configurations. Relative to the
Expand Down Expand Up @@ -392,11 +355,11 @@ checkmk_server__multisite_users: '{{ checkmk_server__multisite_debops_users |
checkmk_server__multisite_debops_users:
ansible:
alias: 'Automation User used by Ansible'
automation_secret: '{{ lookup("password", secret + "/credentials/" + ansible_fqdn + "/checkmk_server/" + checkmk_server__site + "/ansible/secret") }}'
automation_secret: '{{ lookup("password", secret + "/credentials/" + inventory_hostname + "/checkmk_server/" + checkmk_server__site + "/ansible/secret") }}'
roles: [ 'api' ]
sitesync:
alias: 'Synchronization User for Multisite'
password: '{{ lookup("password", secret + "/credentials/" + ansible_fqdn + "/checkmk_server/" + checkmk_server__site + "/sitesync/password") }}'
password: '{{ lookup("password", secret + "/credentials/" + inventory_hostname + "/checkmk_server/" + checkmk_server__site + "/sitesync/password") }}'
roles: [ 'admin' ]

# ]]]
Expand Down Expand Up @@ -450,32 +413,35 @@ checkmk_server__multisite_user_connection_defaults:
#
# Distributed monitoring sites configuration. For more details see
# :ref:`checkmk_server__ref_distributed_sites`
checkmk_server__distributed_sites: {}
checkmk_server__distributed_sites: []
# name: site_name
# inventory_host: slavehost
# tls

# ]]]
# .. envvar:: checkmk_server__sites [[[
#
# List of expanded sites configuration used for distributed sites setup.
checkmk_server__sites: '{{ lookup("template", "lookup/checkmk_server__sites.j2", convert_data=False) | from_yaml }}'

# ]]]
# .. envvar:: checkmk_server__distributed_sites_defaults [[[
#
# Default sites properties for distributed monitoring.
checkmk_server__distributed_sites_defaults:
username: 'sitesync'
password: '{{ lookup("password", secret + "/credentials/" + ansible_fqdn + "/checkmk_server/" + checkmk_server__site + "/sitesync/password") }}'
disabled: False
disable_wato: True
insecure: False
multisiteurl: ''
persist: False
replicate_ec: False
replicate_mkps: True
replication: ''
status_host: None
timeout: 10
url_prefix: ''
user_login: True
# ]]]
# ]]]
# Monitoring Rules [[[
# --------------------
multisite_username: 'sitesync'
#password: '{{ lookup("password", secret + "/credentials/" + ansible_fqdn + "/checkmk_server/" + checkmk_server__site + "/sitesync/password") }}'
multisite_disable_wato: True
multisite_insecure: False
multisite_persist: False
multisite_replicate_ec: False
multisite_replicate_mkps: True
multisite_replication: 'slave'
multisite_status_host: None
multisite_timeout: 10
multisite_user_login: True

# ]]]
# .. envvar:: checkmk_server__site_config_path [[[
#
# Configuration path for Check_MK main configurations. Relative to the site's
Expand Down Expand Up @@ -632,63 +598,15 @@ checkmk_server__site_cfg_netif_description:
checkmk_server__site_packages: []
# ]]]
# ]]]
# PKI Configuration [[[
# ---------------------

# .. envvar:: checkmk_server__pki [[[
#
# Enable or disable support for HTTPS in Check_MK server (using
# debops.pki_).
checkmk_server__pki: '{{ (True
if (ansible_local|d() and ansible_local.pki|d() and
ansible_local.pki.enabled|d() | bool)
else False) | bool }}'

# ]]]
# .. envvar:: checkmk_server__pki_path [[[
#
# Base path for PKI directory.
checkmk_server__pki_path: '{{ ansible_local.pki.path
if (ansible_local|d() and ansible_local.pki|d() and
ansible_local.pki.path|d())
else "/etc/pki/realms" }}'

# ]]]
# .. envvar:: checkmk_server__pki_realm [[[
#
# Default PKI realm used by Check_MK server.
checkmk_server__pki_realm: '{{ ansible_local.pki.realm
if (ansible_local|d() and ansible_local.pki|d() and
ansible_local.pki.realm|d())
else "domain" }}'
# Configuration for other Ansible roles [[[
# -----------------------------------------

# ]]]
# .. envvar:: checkmk_server__pki_ca [[[
#
# Root CA certificate, relative to :envvar:`checkmk_server__pki_realm`.
checkmk_server__pki_ca: 'CA.crt'

# ]]]
# .. envvar:: checkmk_server__pki_crt [[[
#
# Host certificate, relative to :envvar:`checkmk_server__pki_realm`.
checkmk_server__pki_crt: 'default.crt'

# ]]]
# .. envvar:: checkmk_server__pki_key [[[
#
# Host private key, relative to :envvar:`checkmk_server__pki_realm`.
checkmk_server__pki_key: 'default.key'

# ]]]
# .. envvar:: checkmk_server__tls_options [[[
# .. envvar:: checkmk_server__apache__dependent_vhosts [[[
#
# Additional Apache mod_ssl options. Valid configuration keys:
# ``SSLCipherSuite``, ``SSLHonorCipherOrder``, ``SSLProtocols``,
# ``SSLStrictSNIVHostCheck``
checkmk_server__tls_options:
SSLHonorCipherOrder: 'On'
SSLCipherSuite: 'ECDH+AESGCM:DH+AESGCM:ECDH+AES256:DH+AES256:ECDH+AES128:DH+AES:ECDH+3DES:DH+3DES:RSA+AESGCM:RSA+AES:RSA+3DES:!aNULL:!MD5:!DSS'
# Configuration for debops.apache_ Ansible role.
checkmk_server__apache__dependent_vhosts:
- name: '{{ checkmk_server__fqdn }}'
by_role: 'debops-contrib.checkmk_server'
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

NIce work! I guess the Apache omd snippet could be disabled and instead be included in checkmk_server__apache__dependent_vhosts to ensure that omd is only available for this vhost.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ya, I thought about that, but I wasn't sure how to properly do this. To add the snipped to the vhost I guess I have to define include: '/omd/apache/*.conf' but how do I get rid of the conf-enabled/zzz_omd.conf?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Something like:

checkmk_server__apache__dependent_snippets:                                                                                                                                                                        
  'zzz_omd':
    enabled: False

should do.

For the vhost, item.include: [ '/omd/apache/*.conf' ] can be tried.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Added the snippet configuration as you suggested. However, it fails with:

TASK [debops.apache : Create conf-available snippets] ********************************************************************************************
[...]
failed: [cmk01] (item={'key': u'zzz_omd', 'value': {u'enabled': False}}) => {
    "failed": true, 
    "item": {
        "key": "zzz_omd", 
        "value": {
            "enabled": false
        }
    }, 
    "msg": "Unable to find 'etc/apache2/conf-available/zzz_omd.conf.j2' in expected paths."
}

I didn't find a successful way to tell the role that this configuration is provided externally. Any suggestion?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

My mistake, excuse me. Try:

checkmk_server__apache__dependent_snippets:                                                                                                                                                                        
  'zzz_omd':
    enabled: False
    type: 'dont-create'

type: 'dont-create' is there specifically for your usecase, as documented 😉
Have you seen the nice documenteion of the role btw? Ref: https://docs.debops.org/en/latest/ansible/roles/ansible-apache/docs/

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ack, my bad. I read the documentation, but not far enough 😉 I had in mind there is only raw and divert. I guess I was a bit confused by the following sentence:

Type: raw

Available when ``item.type`` is set to ``raw`` or ``divert``.

# ]]]
# ]]]
# ]]]
19 changes: 16 additions & 3 deletions docs/playbooks/checkmk_server.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,16 +6,29 @@

roles:

- role: debops.apache/env
tags: [ 'role::apache', 'role::apache:env' ]

- role: debops-contrib.checkmk_server/env
tags: [ 'role::checkmk_server', 'role::checkmk_server:env' ]

- role: debops.etc_services
tags: [ 'role::etc_services' ]
etc_services__dependent_list:
- '{{ checkmk_server__etc_services__dependent_list }}'
when: checkmk_server__multisite_livestatus|d()
etc_services__dependent_list: '{{ ansible_local.checkmk_server | map(attribute="dependent_vars.etc_services__dependent_list") | list }}'

- role: debops.ferm
tags: [ 'role::ferm' ]
ferm__dependent_rules:
- '{{ apache__ferm__dependent_rules }}'
- '{{ checkmk_server__ferm_dependent_rules }}'

- role: debops.apache
tags: [ 'role::apache' ]
apache__dependent_vhosts: '{{ checkmk_server__apache__dependent_vhosts }}'

- role: debops.users
tags: [ 'role::users' ]
users__dependent_accounts: '{{ ansible_local.checkmk_server | map(attribute="dependent_vars.users__dependent_accounts") | list }}'

- role: debops-contrib.checkmk_server
tags: [ 'role::checkmk_server' ]
1 change: 1 addition & 0 deletions env/defaults
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../defaults
0 ...-1.2.8-read-X-Forwarded-Port-header.patch → ...-1.2.8-read-X-Forwarded-Port-header.patch
View file
Open in desktop
File renamed without changes.
0 ...mk-raw-1.2.8-set-https-proxy-header.patch → ...mk-raw-1.2.8-set-https-proxy-header.patch
View file
Open in desktop
File renamed without changes.
0 ....2.8p4-read-X-Forwarded-Port-header.patch → ....2.8p4-read-X-Forwarded-Port-header.patch
View file
Open in desktop
File renamed without changes.
14 changes: 14 additions & 0 deletions env/tasks/create_sites.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
---
# IMPORTANT:
# These tasks are run for each Check_MK site defined
# in `checkmk_server__sites`. This means they can run multiple
# times per server. If the monitoring site is a remote slave,
# they might even run on a different server. The site configuration
# is available through `site_item`.

- name: Create Check_MK site
command: omd create '{{ site_item.name }}'
args:
creates: '{{ site_item.home }}/etc/omd/site.conf'
delegate_to: '{{ site_item.delegate_to
if (not site_item.delegate_to == inventory_hostname) else omit }}'
1 change: 1 addition & 0 deletions env/tasks/facts.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -0,0 +1 @@
../../tasks/facts.yml
Loading