Categories
DevOps

How to execute multiple Ansible tasks once per group

Execute multiple Ansible tasks once per group and use a dedicated variable to store custom data.

Define hosts that will contain multiple application clusters.

$ cat hosts 
[application:children]
application_cluster_development
application_cluster_production

[application_cluster_development]
dev_server_1 ansible_host=127.0.0.1
dev_server_2 ansible_host=127.0.0.1
dev_server_3 ansible_host=127.0.0.1

[application_cluster_production]
prod_server1 ansible_host=127.0.0.1
prod_server2 ansible_host=127.0.0.1
prod_server3 ansible_host=127.0.0.1

Define per cluster configuration.

$ cat group_vars/application_cluster_production/application_cluster.yml 
---

cluster_items:
  - name: Alpha
    state: present
  - name: Beta
    state: present
  - name: Gamma
    state: present
  - name: Delta
    state: present
  - name: Epsilon
    state: absent

Define Ansible playbook.

$ cat playbook.yml 
---
- hosts: application
  tasks:
    - name: Determine application cluster group
      set_fact:
        application_cluster_group: '{{ group_names | select("match","application_cluster_*") | first }}'

    - block: 
      - name: Define per cluster variable # per cluster short-lived variable
        set_fact:
          per_cluster_variable: '{{ per_cluster_variable | default({}) | 
                                    combine({inventory_hostname: {
                                      "cluster_items":         cluster_items|default([]), 
                                      "cluster_group":         application_cluster_group, 
                                      "cluster_group_members": groups[application_cluster_group]},
                                      "cluster_hostname":      inventory_hostname 
                                    }) 
                                 }}'

      - name: Display per cluster variable
        debug:
          var: per_cluster_variable[inventory_hostname]

      - name: List cluster items that should exist
        debug:
          var: item
        loop: '{{ per_cluster_variable[inventory_hostname]["cluster_items"] }}'
        when:
          - item.state == "present"

      - name: List cluster items that should not exist
        debug:
          var: item
        loop: '{{ per_cluster_variable[inventory_hostname]["cluster_items"] }}'
        when:
          - item.state == "absent"

      when: inventory_hostname == groups[application_cluster_group][0] # execute this block once per group

Execute created playbook to see how it works.

$ ansible-playbook -i hosts playbook.yml
                                        
PLAY [application] *********************************************************************************************************************************************
                                        
TASK [Gathering Facts] *****************************************************************************************************************************************
ok: [prod_server2]      
ok: [dev_server_2]             
ok: [dev_server_3]                
ok: [prod_server1]
ok: [dev_server_1]      
ok: [prod_server3]                                                              
                                        
TASK [Determine application cluster group] *********************************************************************************************************************
ok: [dev_server_1]      
ok: [dev_server_2]              
ok: [dev_server_3]                
ok: [prod_server1]                                                              
ok: [prod_server2]                                                              
ok: [prod_server3]                                                              
                                                                                
TASK [Define per cluster variable] *****************************************************************************************************************************
ok: [dev_server_1]             
skipping: [dev_server_2]
skipping: [dev_server_3]  
ok: [prod_server1]       
skipping: [prod_server2]
skipping: [prod_server3]                
                                                                                                                                                                
TASK [Display per cluster variable] ****************************************************************************************************************************
ok: [dev_server_1] => {                                                                                                                                         
    "per_cluster_variable[inventory_hostname]": {                                                                                                               
        "cluster_group": "application_cluster_development",                                                                                                     
        "cluster_group_members": [                                                                                                                              
            "dev_server_1",                                                                                                                                     
            "dev_server_2",                                                                                                                                     
            "dev_server_3"                                                                                                                                      
        ],
        "cluster_items": []
    }                                                                           
}                              
skipping: [dev_server_2]
skipping: [dev_server_3]
ok: [prod_server1] => {
    "per_cluster_variable[inventory_hostname]": {
        "cluster_group": "application_cluster_production",
        "cluster_group_members": [
            "prod_server1",
            "prod_server2",
            "prod_server3"
        ],
        "cluster_items": [
            {
                "name": "Alpha",
                "state": "present"
            },
            {
                "name": "Beta",
                "state": "present"
            },
            {
                "name": "Gamma",
                "state": "present"
            },
            {
                "name": "Delta",
                "state": "present"
            },
            {
                "name": "Epsilon",
                "state": "absent"
            }
        ]
    }
}
skipping: [prod_server2]
skipping: [prod_server3]

TASK [List cluster items that should exist] ********************************************************************************************************************
skipping: [dev_server_2]
skipping: [dev_server_3]
ok: [prod_server1] => (item={'name': 'Alpha', 'state': 'present'}) => {
    "ansible_loop_var": "item",
    "item": {
        "name": "Alpha",
        "state": "present"
    }
}
skipping: [prod_server2]
ok: [prod_server1] => (item={'name': 'Beta', 'state': 'present'}) => {                                                                                          
    "ansible_loop_var": "item",
    "item": {
        "name": "Beta",
        "state": "present"
    }
}
ok: [prod_server1] => (item={'name': 'Gamma', 'state': 'present'}) => {
    "ansible_loop_var": "item",
    "item": {
        "name": "Gamma",
        "state": "present"
    }
}
ok: [prod_server1] => (item={'name': 'Delta', 'state': 'present'}) => {
    "ansible_loop_var": "item",
    "item": {
        "name": "Delta",
        "state": "present"
    }
}
skipping: [prod_server3]
skipping: [prod_server1] => (item={'name': 'Epsilon', 'state': 'absent'}) 

TASK [List cluster items that should not exist] ****************************************************************************************************************
skipping: [dev_server_2]
skipping: [dev_server_3]
skipping: [prod_server2]
skipping: [prod_server1] => (item={'name': 'Alpha', 'state': 'present'}) 
skipping: [prod_server1] => (item={'name': 'Beta', 'state': 'present'}) 
skipping: [prod_server1] => (item={'name': 'Gamma', 'state': 'present'}) 
skipping: [prod_server1] => (item={'name': 'Delta', 'state': 'present'}) 
ok: [prod_server1] => (item={'name': 'Epsilon', 'state': 'absent'}) => {
    "ansible_loop_var": "item",
    "item": {
        "name": "Epsilon",
        "state": "absent"
    }
}
skipping: [prod_server3]

PLAY RECAP *****************************************************************************************************************************************************
dev_server_1               : ok=4    changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
dev_server_2               : ok=2    changed=0    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0   
dev_server_3               : ok=2    changed=0    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0   
prod_server1               : ok=6    changed=0    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
prod_server2               : ok=2    changed=0    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0   
prod_server3               : ok=2    changed=0    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0   

Additional notes

I keep things simpler then presented in the above example, as a basic list accessed using per_cluster_variable[inventory_hostname] variable is more then enough.

per_cluster_variable: '{{ per_cluster_variable | default({}) | combine({inventory_hostname: cluster_items|default([]) }) }}'

This technique is very useful when you use Ansible to provision multiple clusters, for example, to keep an index list for Elasticsearch or a topic list for Kafka, …