Shell scripts in Ansible
Suppose a customer has asked you to help with migrating a script to deploy a centralized sudoers file on RHEL and AIX servers.
Well, this is quite a common scenario, and by its example we can demonstrate the use of advanced Ansible capabilities, as well as how the approach is changing - from a script that performs a certain task to idempotent (without making changes) description and monitoring of the instance state.
Take the script:
')
There are 212 lines of code, with no version control in the sudoers file. The customer already has a process that runs once a week and checks the checksum of the file to ensure security. Although the script has a reference to Solaris, for this customer we did not have to transfer this requirement as well.
Let's start by creating a role and putting the sudoers file in Git for version control. Among other things, this will allow us to get rid of the need to mount NFS volumes.
With the “validate” and “backup” options for the copy and template modules, we can get rid of the need to write code to create backup copies and restore the file. In this case, the validation is performed before the file is placed at the destination, and if the validation fails, the module generates an error.
For each role we need to specify the tasks, patterns and variables. Here is the structure of the corresponding file:
The role play (playbook) file , sudoers.yml , has a simple structure:
Role variables are located in the vars / main.yml file . Here is the checksum file and include / exclude directives that will be used to create special logic to skip the “Lawson” hosts and include the sudoers.d file only on the “hd” hosts.
Here is the contents of the vars / main.yml file :
If we use the copy and lineinfile modules , the role will not be idempotent. The copy module will install the base file, and the lineinfile will re-insert the include each time it runs. Since this role will run on the Ansible Tower , idempotency is imperative. We convert the file to a jinja2 template.
In the first line, we add the following command to control spaces and indents :
Please note that newer versions of the template module include trim_blocks options (added in Ansible 2.4).
Here is the code that inserts the include line at the end of the file:
We use the conditional construct ({% if%}, {% endif%}) for the shell command that inserts a string for hosts whose names begin with the characters "hd". We use the facts Ansible and filter [0: 2] to parse the host name.
Now go to the tasks. First, you need to set a fact for parsing the host name. We will use the fact of “parhost” in the conditional construction.
There is no csum parameter on the RHEL stock server. If necessary, we can use another fact to conditionally indicate the name of the binary file with the checksum. Note that additional code may be required if these functions differ on AIX, Solaris, and Linux.
In addition, it is necessary to resolve the issue of differences in the root groups on AIX and RHEL.
The use of blocks (block) will allow us to set the condition for the entire task. We will use the condition at the end of the block to exclude the “la” hosts.
The template module validates and installs a file. We fix the result so that it is possible to determine whether the task has changed. Using the validate parameter in this module allows you to make sure that the new sudoer file is valid before placing it on the host.
If a new template has been installed, run a shell script to generate a file with a checksum. The conditional construct updates the checksum file when you install the sudoers template, or if the checksum file is missing. Since the running process also tracks other files, we use the shell code presented in the source script:
The file module checks the installation of the required permissions:
Since the backup parameter does not provide any options for processing previous backups, we will have to take care of creating the appropriate code ourselves. In the example below, we use the "register" parameter and the "stdout_lines" field for this.
Block Completion:
The intended use case is to launch this role on the Ansible Tower. Ansible Tower alerts can be configured so that, in the event of a failure in the execution of a task, alerts come to email, on Slack, or in some other way. This role is launched in Ansible, Ansible Engine or Ansible Tower.
As a result, we have removed all unnecessary from the script and created a completely idempotent role that can provide the desired state of the sudoers file. Using SCM allows for version control, more efficient change management and transparency. CI / CD with Jenkins or other tools allow you to set up automated testing of Ansible code for future changes. The role of Auditor in Ansible Tower allows you to monitor and ensure compliance with the requirements of organizations.
You could remove the code for working with checksums from the script, but for this the customer would have to first consult with their security service. If necessary, the sudoers template can be protected with Ansible Vault. Finally, using groups allows you to avoid writing logic using includes and excludes.
→ You can upload a role from GitHub at this link.
Well, this is quite a common scenario, and by its example we can demonstrate the use of advanced Ansible capabilities, as well as how the approach is changing - from a script that performs a certain task to idempotent (without making changes) description and monitoring of the instance state.
Take the script:
')
#!/bin/sh # Desc: Distribute unified copy of /etc/sudoers # # $Id: $ #set -x export ODMDIR=/etc/repos # # perform any cleanup actions we need to do, and then exit with the # passed status/return code # clean_exit() { cd / test -f "$tmpfile" && rm $tmpfile exit $1 } #Set variables PROG=`basename $0` PLAT=`uname -s|awk '{print $1}'` HOSTNAME=`uname -n | awk -F. '{print $1}'` HOSTPFX=$(echo $HOSTNAME |cut -c 1-2) NFSserver="nfs-server" NFSdir="/NFS/AIXSOFT_NFS" MOUNTPT="/mnt.$$" MAILTO="unix@company.com" DSTRING=$(date +%Y%m%d%H%M) LOGFILE="/tmp/${PROG}.dist_sudoers.${DSTRING}.log" BKUPFILE=/etc/sudoers.${DSTRING} SRCFILE=${MOUNTPT}/skel/sudoers-uni MD5FILE="/.sudoers.md5" echo "Starting ${PROG} on ${HOSTNAME}" >> ${LOGFILE} 2>&1 # Make sure we run as root runas=`id | awk -F'(' '{print $1}' | awk -F'=' '{print $2}'` if [ $runas -ne 0 ] ; then echo "$PROG: you must be root to run this script." >> ${LOGFILE} 2>&1 exit 1 fi case "$PLAT" in SunOS) export PINGP=" -t 7 $NFSserver " export MOUNTP=" -F nfs -o vers=3,soft " export PATH="/usr/sbin:/usr/bin" echo "SunOS" >> ${LOGFILE} 2>&1 exit 0 ;; AIX) export PINGP=" -T 7 $NFSserver 2 2" export MOUNTP=" -o vers=3,bsy,soft " export PATH="/usr/bin:/etc:/usr/sbin:/usr/ucb:/usr/bin/X11:/sbin:/usr/java5/jre/bin:/usr/java5/bin" printf "Continuing on AIX...\n\n" >> ${LOGFILE} 2>&1 ;; Linux) export PINGP=" -t 7 -c 2 $NFSserver" export MOUNTP=" -o nfsvers=3,soft " export PATH="/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin:/root/bin" printf "Continuing on Linux...\n\n" >> ${LOGFILE} 2>&1 ;; *) echo "Unsupported Platform." >> ${LOGFILE} 2>&1 exit 1 esac ## ## Exclude Lawson Hosts ## if [ ${HOSTPFX} = "la" ] then echo "Exiting Lawson host ${HOSTNAME} with no changes." >> ${LOGFILE} 2>&1 exit 0 fi ## ## * NFS Mount Section * ## ## Check to make sure NFS host is up printf "Current PATH is..." >> ${LOGFILE} 2>&1 echo $PATH >> $LOGFILE 2>&1 ping $PINGP >> $LOGFILE 2>&1 if [ $? -ne 0 ]; then echo " NFS server is DOWN ... ABORTING SCRIPT ... Please check server..." >> $LOGFILE echo "$PROG failed on $HOSTNAME ... NFS server is DOWN ... ABORTING SCRIPT ... Please check server ... " | mailx -s "$PROG Failed on $HOSTNAME" $MAILTO exit 1 else echo " NFS server is UP ... We will continue..." >> $LOGFILE fi ## ## Mount NFS share to HOSTNAME. We do this using a soft mount in case it is lost during a backup ## mkdir $MOUNTPT mount $MOUNTP $NFSserver:${NFSdir} $MOUNTPT >> $LOGFILE 2>&1 ## ## Check to make sure mount command returned 0. If it did not odds are something else is mounted on /mnt.$$ ## if [ $? -ne 0 ]; then echo " Mount command did not work ... Please check server ... Odds are something is mounted on $MOUNTPT ..." >> $LOGFILE echo " $PROG failed on $HOSTNAME ... Mount command did not work ... Please check server ... Odds are something is mounted on $MOUNTPT ..." | mailx -s "$PROG Failed on $HOSTNAME" $MAILTO exit 1 else echo " Mount command returned a good status which means $MOUNPT was free for us to use ... We will now continue ..." >> $LOGFILE fi ## ## Now check to see if the mount worked ## if [ ! -f ${SRCFILE} ]; then echo " File ${SRCFILE} is missing... Maybe NFS mount did NOT WORK ... Please check server ..." >> $LOGFILE echo " $PROG failed on $HOSTNAME ... File ${SRCFILE} is missing... Maybe NFS mount did NOT WORK ... Please check server ..." | mailx -s "$PROG Failed on $HOSTNAME" $MA ILTO umount -f $MOUNTPT >> $LOGFILE rmdir $MOUNTPT >> $LOGFILE exit 1 else echo " NFS mount worked we are going to continue ..." >> $LOGFILE fi ## ## * Main Section * ## if [ ! -f ${BKUPFILE} ] then cp -p /etc/sudoers ${BKUPFILE} else echo "Backup file already exists$" >> ${LOGFILE} 2>&1 exit 1 fi if [ -f "$SRCFILE" ] then echo "Copying in new sudoers file from $SRCFILE." >> ${LOGFILE} 2>&1 cp -p $SRCFILE /etc/sudoers chmod 440 /etc/sudoers else echo "Source file not found" >> ${LOGFILE} 2>&1 exit 1 fi echo >> ${LOGFILE} 2>&1 visudo -c |tee -a ${LOGFILE} if [ $? -ne 0 ] then echo "sudoers syntax error on $HOSTNAME." >> ${LOGFILE} 2>&1 mailx -s "${PROG}: sudoers syntax error on $HOSTNAME" "$MAILTO" << EOF Syntax error /etc/sudoers on $HOSTNAME. Reverting changes Please investigate. EOF echo "Reverting changes." >> ${LOGFILE} 2>&1 cp -p ${BKUPFILE} /etc/sudoers else # # Update checksum file # grep -v '/etc/sudoers' ${MD5FILE} > ${MD5FILE}.tmp csum /etc/sudoers >> ${MD5FILE}.tmp mv ${MD5FILE}.tmp ${MD5FILE} chmod 600 ${MD5FILE} fi echo >> ${LOGFILE} 2>&1 if [ "${HOSTPFX}" = "hd" ] then printf "\nAppending #includedir /etc/sudoers.d at end of file.\n" >> ${LOGFILE} 2>&1 echo "" >> /etc/sudoers echo "## Read drop-in files from /etc/sudoers.d (the # here does not mean a comment)" >> /etc/sudoers echo "#includedir /etc/sudoers.d" >> /etc/sudoers fi ## ## * NFS Un-mount Section * ## ## ## Unmount /mnt.$$ directory ## umount ${MOUNTPT} >> $LOGFILE 2>&1 if [ -d ${MOUNTPT} ]; then rmdir ${MOUNTPT} >> $LOGFILE 2>&1 fi ## ## Make sure that /mnt.$$ got unmounted ## if [ -f ${SRCFILE} ]; then echo " The umount command failed to unmount ${MOUNTPT} ... We will not force the unmount ..." >> $LOGFILE umount -f ${MOUNTPT} >> $LOGFILE 2>&1 if [ -d ${MOUNTPT} ]; then rmdir ${MOUNTPT} >> $LOGFILE 2>&1 fi else echo " $MOUNTPT was unmounted ... There is no need for user intervention on $HOSTNAME ..." >> $LOGFILE fi # # as always, exit cleanly # clean_exit 0 There are 212 lines of code, with no version control in the sudoers file. The customer already has a process that runs once a week and checks the checksum of the file to ensure security. Although the script has a reference to Solaris, for this customer we did not have to transfer this requirement as well.
Let's start by creating a role and putting the sudoers file in Git for version control. Among other things, this will allow us to get rid of the need to mount NFS volumes.
With the “validate” and “backup” options for the copy and template modules, we can get rid of the need to write code to create backup copies and restore the file. In this case, the validation is performed before the file is placed at the destination, and if the validation fails, the module generates an error.
For each role we need to specify the tasks, patterns and variables. Here is the structure of the corresponding file:
The role play (playbook) file , sudoers.yml , has a simple structure:
---
##
# Role playbook
##
- hosts: all
roles:
- sudoers
...
Role variables are located in the vars / main.yml file . Here is the checksum file and include / exclude directives that will be used to create special logic to skip the “Lawson” hosts and include the sudoers.d file only on the “hd” hosts.
Here is the contents of the vars / main.yml file :
---
MD5FILE: /root/.sudoer.md5
EXCLUDE: la
INCLUDE: hd
...
If we use the copy and lineinfile modules , the role will not be idempotent. The copy module will install the base file, and the lineinfile will re-insert the include each time it runs. Since this role will run on the Ansible Tower , idempotency is imperative. We convert the file to a jinja2 template.
In the first line, we add the following command to control spaces and indents :
#jinja2: lstrip_blocks: True, trim_blocks: True
Please note that newer versions of the template module include trim_blocks options (added in Ansible 2.4).
Here is the code that inserts the include line at the end of the file:
{% if ansible_hostname[0:2] == INCLUDE %}
#includedir /etc/sudoers.d
{% endif %}
We use the conditional construct ({% if%}, {% endif%}) for the shell command that inserts a string for hosts whose names begin with the characters "hd". We use the facts Ansible and filter [0: 2] to parse the host name.
Now go to the tasks. First, you need to set a fact for parsing the host name. We will use the fact of “parhost” in the conditional construction.
---
##
# Parse hostnames to grab 1st 2 characters
##
- name: "Parse hostname's 1st 2 characters"
set_fact: parhost={{ ansible_hostname[0:2] }}
There is no csum parameter on the RHEL stock server. If necessary, we can use another fact to conditionally indicate the name of the binary file with the checksum. Note that additional code may be required if these functions differ on AIX, Solaris, and Linux.
In addition, it is necessary to resolve the issue of differences in the root groups on AIX and RHEL.
##
# Conditionally set name of checksum binary
##
- name: "set checksum binary"
set_fact:
csbin: "{{ 'cksum' if (ansible_distribution == 'RedHat') else 'csum' }}"
##
# Conditionally set name of root group
##
- name: "set system group"
set_fact:
sysgroup: "{{ 'root' if (ansible_distribution == 'RedHat') else 'sys' }}"
The use of blocks (block) will allow us to set the condition for the entire task. We will use the condition at the end of the block to exclude the “la” hosts.
##
# Enclose in block so we can use parhost to exclude hosts
##
- block:
The template module validates and installs a file. We fix the result so that it is possible to determine whether the task has changed. Using the validate parameter in this module allows you to make sure that the new sudoer file is valid before placing it on the host.
##
# Validate will prevent bad files, no need to revert
# Jinja2 template will add include line
##
- name: Ensure sudoers file
template:
src: sudoers.j2
dest: /etc/sudoers
owner: root
group: "{{ sysgroup }}"
mode: 0440
backup: yes
validate: /usr/sbin/visudo -cf %s
register: sudochg
If a new template has been installed, run a shell script to generate a file with a checksum. The conditional construct updates the checksum file when you install the sudoers template, or if the checksum file is missing. Since the running process also tracks other files, we use the shell code presented in the source script:
- name: sudoers checksum
shell: "grep -v '/etc/sudoers' {{ MD5FILE }} > {{ MD5FILE }}.tmp ; {{ csbin }} /etc/sudoers >> {{ MD5FILE }} ; mv {{ MD5FILE }}.tmp {{ MD5FILE }}"
when: sudochg.changed or MD5STAT.exists == false
The file module checks the installation of the required permissions:
- name: Ensure MD5FILE permissions
file:
path: "{{ MD5FILE }}"
owner: root
group: "{{ sysgroup }}"
mode: 0600
state: file
Since the backup parameter does not provide any options for processing previous backups, we will have to take care of creating the appropriate code ourselves. In the example below, we use the "register" parameter and the "stdout_lines" field for this.
##
# List and clean up backup files. Retain 3 copies.
##
- name: List /etc/sudoers.*~ files
shell: "ls -t /etc/sudoers*~ |tail -n +4"
register: LIST_SUDOERS
changed_when: false
- name: Cleanup /etc/sudoers.*~ files
file:
path: "{{ item }}"
state: absent
loop: "{{ LIST_SUDOERS.stdout_lines }}"
when: LIST_SUDOERS.stdout_lines != ""
Block Completion:
##
# This conditional restricts what hosts this block runs on
##
when: parhost != EXCLUDE
...
The intended use case is to launch this role on the Ansible Tower. Ansible Tower alerts can be configured so that, in the event of a failure in the execution of a task, alerts come to email, on Slack, or in some other way. This role is launched in Ansible, Ansible Engine or Ansible Tower.
As a result, we have removed all unnecessary from the script and created a completely idempotent role that can provide the desired state of the sudoers file. Using SCM allows for version control, more efficient change management and transparency. CI / CD with Jenkins or other tools allow you to set up automated testing of Ansible code for future changes. The role of Auditor in Ansible Tower allows you to monitor and ensure compliance with the requirements of organizations.
You could remove the code for working with checksums from the script, but for this the customer would have to first consult with their security service. If necessary, the sudoers template can be protected with Ansible Vault. Finally, using groups allows you to avoid writing logic using includes and excludes.
→ You can upload a role from GitHub at this link.
Source: https://habr.com/ru/post/433792/
All Articles