Merge pull request #213 from michaelpporter/client_templates

Templates in client_definitions

.gitignore

@@ -1,3 +1,4 @@
 site
-molecule/default/data/
+molecule/shared/data/*
+!molecule/shared/data/static/
 molecule/*/cache/

CHANGELOG.md

@@ -5,6 +5,8 @@ This project adheres to [Semantic Versioning](http://semver.org/)
 The format is based on [Keep a Changelog](http://keepachangelog.com/).
 
 ## [Unreleased]
+- Add `client_templates` option for group based tempaltes (@michaelpporter)
+- Add `run_once: true` to `delegate_to: localhost` (@michaelpporter)
 
 ## [5.0.0] - 2019-02-19
 ### Breaking Changes

docs/dynamic_checks.md

@@ -15,12 +15,14 @@ data/static
     |-- handlers
     `-- mutators
 ```
-With all the checks and definitions dropped in, the static data store might look something like this:
+With all the checks (check commands) and definitions (JSON check files) dropped in, the static data store might look something like this:
 ```
 $ tree data/static
 data/static
 `-- sensu
     |-- checks
+    |   |-- mysql
+    |   |   `-- check_mysql.sh
     |   |-- sensu_rabbitmq_servers
     |   |   `-- check_rabbitmq.sh
     |   |-- sensu_redis_servers
@@ -42,6 +44,11 @@ data/static
     |   `-- smartos_check_mem.json.j2
     |-- client_definitions
     |   |-- sensu_rabbitmq_servers
+    |   |   `-- check_users.json
+    |   `-- webservers
+    |       `-- check_uptime.json
+    |-- client_templates
+    |   |-- mysql
     |   |   `-- check_users.json.j2
     |   `-- webservers
     |       `-- check_uptime.json.j2
@@ -49,7 +56,7 @@ data/static
     |   `-- pushover.rb
     `-- mutators
 ```
-As you can see, in the `sensu/checks` directory, there are the `sensu_rabbitmq_servers`, `sensu_redis_servers`, `webservers` & `zones` subdirectories.
+As you can see, in the `sensu/checks` directory, there are the `mysql`, `sensu_rabbitmq_servers`, `sensu_redis_servers`, `webservers` & `zones` subdirectories.
 If you've had a peruse through some of the other documentation here, you'll know that these groups are defined within my Ansible inventory:
 ``` ini
 [sensu_rabbitmq_servers]
@@ -69,6 +76,9 @@ tater.cmacr.ae
 beardy.cmacr.ae
 pkgsrc.cmacr.ae
 
+[mysql]
+mysql.cmacr.ae
+
 [zones]
 ansible.cmacr.ae
 beardy.cmacr.ae
@@ -83,6 +93,7 @@ sensu.cmacr.ae
 tater.cmacr.ae
 web.cmacr.ae
 test.cmacr.ae
+mysql.cmacr.ae
 ```
 Under these subdirectories, you can see [checks](https://docs.sensu.io/sensu-core/latest/reference/checks/) that relate to the directory they're placed in.
 For example, our `webservers` subdirectory includes a `check_nginx.sh` script, whilst the `sensu_rabbitmq_servers` subdirectory has one that most likely checks for RabbitMQ problems (it does... trust me).
@@ -110,26 +121,69 @@ This will [register](https://docs.ansible.com/ansible/latest/user_guide/playbook
 
 And, because nodes can of course be members of more than just one group, checks will be deployed in full to nodes that belong to several groups!
 
-Additionally, standalone checks can be distributed to hosts based on group membership. These definitions are located in the client_definitions folder. These will be deployed to the configuration directory of the clients.
+Additionally, standalone checks and subscription cheecks (if your client is in [safe_mode](https://docs.sensu.io/sensu-core/latest/reference/clients/)), can be distributed to hosts based on group membership. These definitions are located in the `client_definitions` folder. These will be deployed to the configuration directory of the clients.
 
 These are deployed with the following pair of plays, also in the `tasks/plugins.yml` playbook:
 ``` yaml
 - name: Register available client definitions
-  local_action: command ls {{ static_data_store }}/sensu/client_definitions
+  command: "ls {{ static_data_store }}/sensu/client_definitions"
+  delegate_to: localhost
   register: sensu_available_client_definitions
   changed_when: false
   become: false
+  run_once: true
 
 - name: Deploy client definitions
   copy:
     src: "{{ static_data_store }}/sensu/client_definitions/{{ item }}/"
-    dest: "{{ sensu_config_path }}/conf.d/{{ item | basename | regex_replace('.j2', '')}}"
-    mode: 0755
+    dest: "{{ sensu_config_path }}/conf.d/{{ item | basename | regex_replace('.j2', '') }}"
+    owner: "{{ sensu_user_name }}"
+    group: "{{ sensu_group_name }}"
+  when:
+    - sensu_available_client_definitions is defined
+    - sensu_available_client_definitions is not skipped
+    - item in sensu_available_client_definitions.stdout_lines
+  loop: "{{ group_names|flatten }}"
+  notify: restart sensu-client service
+```
+
+If you are using templates for your, standalone checks and subscription cheecks can be distributed from the  `client_templates` folder. These will be deployed to the configuration directory of the clients. **Note** if you have a matching folder/file name in the `client_definitions` they will be overwritten by the template.
+
+These are deployed with the following set of plays, also in the `tasks/plugins.yml` playbook:
+
+``` yaml
+- name: Register available client templates
+  command: "ls {{ static_data_store }}/sensu/client_templates"
+  delegate_to: localhost
+  register: sensu_available_client_templates
+  changed_when: false
+  become: false
+  run_once: true
+
+- name: Deploy client template folders
+  file:
+    path: '{{ sensu_config_path }}/conf.d/{{ item | basename }}'
+    state: directory
     owner: "{{ sensu_user_name }}"
     group: "{{ sensu_group_name }}"
-  when: "sensu_available_client_definitions is defined and item in sensu_available_client_definitions.stdout_lines"
+  when:
+    - sensu_available_client_templates is defined
+    - sensu_available_client_templates is not skipped
+    - item in sensu_available_client_templates.stdout_lines
   loop: "{{ group_names|flatten }}"
   notify: restart sensu-client service
+
+- name: Deploy client templates
+  template:
+    src: "{{ static_data_store }}/sensu/client_templates/{{ item.path | dirname }}/{{ item.path | basename }}"
+    dest: "{{ sensu_config_path }}/conf.d/{{ item.path | dirname }}/{{ item.path | basename | regex_replace('.j2', '') }}"
+    owner: "{{ sensu_user_name }}"
+    group: "{{ sensu_group_name }}"
+  with_filetree: "{{ static_data_store }}/sensu/client_templates"
+  when:
+    - item.state == 'file'
+    - item.path | dirname in group_names
+  notify: restart sensu-client service
 ```
 
 ## Picking up changes on the fly

docs/index.md

@@ -1,7 +1,9 @@
 # Ansible Sensu [![Ansible Galaxy](https://img.shields.io/badge/galaxy-sensu.sensu-660198.svg?style=flat)](https://galaxy.ansible.com/sensu/sensu/)
+
 An [Ansible](https://ansible.com) role that deploys a full [Sensu](https://sensuapp.org) stack, a modern, open source monitoring framework.
 
 ## Features
+
 - Deploy a full [Sensu](https://sensu.io) stack, including RabbitMQ, redis, and the [Uchiwa dashboard](https://uchiwa.io/)
 - Tight integration with the Ansible inventory - deployment of monitoring checks based on inventory grouping
 - Fine grained control over dynamic client configurations
@@ -14,9 +16,11 @@ Along with deploying the Sensu Server, API and clients, this role can deploy a f
 However, if you want to rely on other roles/management methods to deploy/manage these services, [it's nice and easy to integrate this role](integration/).
 
 ## Requirements
+
 This role requires Ansible 2.5
 
 ## Supported Platforms
+
 ### Automatically tested via TravisCI
 
 - [CentOS - 6](https://wiki.centos.org/Manuals/ReleaseNotes/CentOS6.9)
@@ -32,15 +36,18 @@ This role requires Ansible 2.5
 - [Amazon Linux 2](https://aws.amazon.com/amazon-linux-2/)
 
 ### Supported manually (compatibility not always guaranteed)
+
 - [SmartOS - base-64 15.x.x](https://docs.joyent.com/images/smartos/base#version-15xx)
 - [FreeBSD - 10.3, 11.0 (64-bit only)](https://www.freebsd.org/releases/10.2R/relnotes.html)
 - [OpenBSD - 6.2](https://www.openbsd.org/62.html)
 
 ## Role Variables
+
 All variables have sensible defaults, which can be found in `defaults/main.yml`.
 Head over to [the role variables page](role_variables.md) to review them
 
 ## Install (Ansible Galaxy)
+
 To install this role from [Ansible Galaxy](https://galaxy.ansible.com), simpy run:
 `ansible-galaxy install sensu.sensu`
 
@@ -51,6 +58,7 @@ To install this role from [Ansible Galaxy](https://galaxy.ansible.com), simpy ru
     roles:
       - role: sensu.sensu
 ```
+
 Or, passing parameter values:
 
 ``` yaml
@@ -58,19 +66,20 @@ Or, passing parameter values:
     roles:
       - role: sensu.sensu
         sensu_master: true
-        sensu_include_dashboard: true 
+        sensu_include_dashboard: true
 ```
 
-License
--------
+## License
+
 [MIT](license.md)
 
-Author Information
-------------------
+## Author Information
+
 Originally created by [Calum MacRae](http://cmacr.ae)
 Supported by the [Sensu Community Ansible Maintainers](https://github.com/sensu-plugins/community/#maintained-areas)
 
 ### Contributors
+
 See the projects [Contributors page](https://github.com/sensu/sensu-ansible/graphs/contributors)
 
 Feel free to:

molecule/amazonlinux/molecule.yml

@@ -13,11 +13,15 @@ platforms:
     command: /sbin/init
     capabilities:
       - SYS_ADMIN
+    groups:
+      - sensu_checks
   - name: amazonlinux-2
     image: dokken/amazonlinux-2
     command: /usr/lib/systemd/systemd
     capabilities:
       - SYS_ADMIN
+    groups:
+      - sensu_checks
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro
 provisioner:
@@ -51,6 +55,9 @@ provisioner:
         sensu_api_host: "{{ ansible_hostname }}"
         ansible_default_ipv4:
           address: 127.0.0.1
+        sensu_remote_plugins:
+          - sensu-plugins-disk-checks
+        sensu_check_interval: 60
     host_vars:
       amazonlinux-1:
         inspec_version: el6

molecule/centos/molecule.yml

@@ -13,11 +13,15 @@ platforms:
     command: /sbin/init
     capabilities:
       - SYS_ADMIN
+    groups:
+      - sensu_checks
   - name: centos-7
     image: dokken/centos-7
     command: /usr/lib/systemd/systemd
     capabilities:
       - SYS_ADMIN
+    groups:
+      - sensu_checks
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro
 provisioner:
@@ -51,6 +55,9 @@ provisioner:
         sensu_api_host: "{{ ansible_hostname }}"
         ansible_default_ipv4:
           address: 127.0.0.1
+        sensu_remote_plugins:
+          - sensu-plugins-disk-checks
+        sensu_check_interval: 60
     host_vars:
       centos-6:
         inspec_version: el6

molecule/debian/molecule.yml

@@ -14,6 +14,8 @@ platforms:
     privileged: yes
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro
+    groups:
+      - sensu_checks
   - name: debian-9
     image: dokken/debian-9
     command: /lib/systemd/systemd
@@ -21,6 +23,8 @@ platforms:
       - SYS_ADMIN
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro
+    groups:
+      - sensu_checks
 provisioner:
   name: ansible
   config_options:
@@ -52,6 +56,9 @@ provisioner:
         sensu_api_host: "{{ ansible_hostname }}"
         ansible_default_ipv4:
           address: 127.0.0.1
+        sensu_remote_plugins:
+          - sensu-plugins-disk-checks
+        sensu_check_interval: 60
     host_vars:
       debian-8:
         inspec_version: ubuntu1604

molecule/fedora/molecule.yml

@@ -15,6 +15,8 @@ platforms:
       - SYS_ADMIN
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro
+    groups:
+      - sensu_checks
   - name: fedora-27
     image: dokken/fedora-27
     command: /usr/lib/systemd/systemd
@@ -22,6 +24,8 @@ platforms:
       - SYS_ADMIN
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro
+    groups:
+      - sensu_checks
   - name: fedora-28
     image: dokken/fedora-latest
     command: /usr/lib/systemd/systemd
@@ -29,6 +33,8 @@ platforms:
       - SYS_ADMIN
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro
+    groups:
+      - sensu_checks
 provisioner:
   name: ansible
   config_options:
@@ -60,6 +66,9 @@ provisioner:
         sensu_api_host: "{{ ansible_hostname }}"
         ansible_default_ipv4:
           address: 127.0.0.1
+        sensu_remote_plugins:
+          - sensu-plugins-disk-checks
+        sensu_check_interval: 60
     host_vars:
       fedora-26:
         inspec_version: el7

molecule/shared/data/static/sensu/client_definitions/sensu_masters/check_disk_usage.json

@@ -0,0 +1,11 @@
+{
+    "checks": {
+        "check_disk_usage": {
+            "command": "check-disk-usage.rb",
+            "standalone": true,
+            "ttl": 1800,
+            "interval": 120,
+            "refresh": 1800
+        }
+    }
+}

molecule/shared/data/static/sensu/client_templates/not_used/not_a_check.json.j2

@@ -0,0 +1,11 @@
+{
+    "checks": {
+        "not_a_check": {
+            "command": "check-disk-usage.rb",
+            "standalone": true,
+            "ttl": 1800,
+            "interval": {{ sensu_check_interval }},
+            "refresh": 1800
+        }
+    }
+}

molecule/shared/data/static/sensu/client_templates/sensu_checks/metrics_disk_usage.json.j2

@@ -0,0 +1,11 @@
+{
+    "checks": {
+        "metrics_disk_usage": {
+            "command": "metrics-disk-usage.rb",
+            "standalone": true,
+            "ttl": 1800,
+            "interval": {{ sensu_check_interval }},
+            "refresh": 1800
+        }
+    }
+}

molecule/shared/tests/test_default.rb

@@ -50,3 +50,22 @@ describe http('http://127.0.0.1:4567/health',
               params: { consumers: 1 }) do
   its('status') { should eq 204 }
 end
+
+# Ensure disk check exists
+describe json('/etc/sensu/conf.d/sensu_masters/check_disk_usage.json') do
+  its(%w[checks check_disk_usage command]) \
+    { should eq 'check-disk-usage.rb' }
+  its(%w[checks check_disk_usage interval]) { should eq 120 }
+end
+
+# Ensure disk metrics exists
+describe json('/etc/sensu/conf.d/sensu_checks/metrics_disk_usage.json') do
+  its(%w[checks metrics_disk_usage command]) \
+    { should eq 'metrics-disk-usage.rb' }
+  its(%w[checks metrics_disk_usage interval]) { should eq 60 }
+end
+
+# Ensure not_used does not exist
+describe file('/etc/sensu/conf.d/not_used/not_a_check.json') do
+  it { should_not exist }
+end

molecule/ubuntu/molecule.yml

@@ -13,11 +13,15 @@ platforms:
     command: /sbin/init
     capabilities:
       - SYS_ADMIN
+    groups:
+      - sensu_checks
   - name: ubuntu-16.04
     image: dokken/ubuntu-16.04
     command: /bin/systemd
     capabilities:
       - SYS_ADMIN
+    groups:
+      - sensu_checks
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro
   - name: ubuntu-18.04
@@ -25,6 +29,8 @@ platforms:
     command: /bin/systemd
     capabilities:
       - SYS_ADMIN
+    groups:
+      - sensu_checks
     volumes:
       - /sys/fs/cgroup:/sys/fs/cgroup:ro
 provisioner:
@@ -58,6 +64,9 @@ provisioner:
         sensu_api_host: "{{ ansible_hostname }}"
         ansible_default_ipv4:
           address: 127.0.0.1
+        sensu_remote_plugins:
+          - sensu-plugins-disk-checks
+        sensu_check_interval: 60
     host_vars:
       ubuntu-14.04:
         inspec_version: ubuntu1404

tasks/plugins.yml

@@ -18,6 +18,7 @@
     dest: "{{ static_data_store }}/sensu/{{ item }}"
   delegate_to: localhost
   become: no
+  run_once: true
   loop:
     - checks
     - filters
@@ -25,6 +26,7 @@
     - mutators
     - definitions
     - client_definitions
+    - client_templates
 
 - name: Ensure any remote plugins defined are present
   shell: umask 0022; sensu-install -p {{ item }}
@@ -38,6 +40,7 @@
   register: sensu_available_checks
   changed_when: false
   become: false
+  run_once: true
 
 - name: Deploy check plugins
   copy:
@@ -100,6 +103,7 @@
   register: sensu_available_client_definitions
   changed_when: false
   become: false
+  run_once: true
 
 - name: Deploy client definitions
   copy:
@@ -109,7 +113,40 @@
     group: "{{ sensu_group_name }}"
   when:
     - sensu_available_client_definitions is defined
-    - sensu_available_checks is not skipped
+    - sensu_available_client_definitions is not skipped
     - item in sensu_available_client_definitions.stdout_lines
   loop: "{{ group_names|flatten }}"
   notify: restart sensu-client service
+
+- name: Register available client templates
+  command: "ls {{ static_data_store }}/sensu/client_templates"
+  delegate_to: localhost
+  register: sensu_available_client_templates
+  changed_when: false
+  become: false
+  run_once: true
+
+- name: Deploy client template folders
+  file:
+    path: '{{ sensu_config_path }}/conf.d/{{ item | basename }}'
+    state: directory
+    owner: "{{ sensu_user_name }}"
+    group: "{{ sensu_group_name }}"
+  when:
+    - sensu_available_client_templates is defined
+    - sensu_available_client_templates is not skipped
+    - item in sensu_available_client_templates.stdout_lines
+  loop: "{{ group_names|flatten }}"
+  notify: restart sensu-client service
+
+- name: Deploy client templates
+  template:
+    src: "{{ static_data_store }}/sensu/client_templates/{{ item.path | dirname }}/{{ item.path | basename }}"
+    dest: "{{ sensu_config_path }}/conf.d/{{ item.path | dirname }}/{{ item.path | basename | regex_replace('.j2', '') }}"
+    owner: "{{ sensu_user_name }}"
+    group: "{{ sensu_group_name }}"
+  with_filetree: "{{ static_data_store }}/sensu/client_templates"
+  when:
+    - item.state == 'file'
+    - item.path | dirname in group_names
+  notify: restart sensu-client service