Basic installation and configuration of Puppet 4 with storage of manifests in SVN

Good day!

Today we will prepare Puppet 4 on Ubuntu Server 16.04 with storage of manifests in SVN. Also, the article will consider an example of creating a simple custom module for installing and configuring the log collection agent in Graylog2 via Graylog Collector Sidecar and using Filebeat as a backend. This example does not claim to be an elegant solution, but describes key aspects with an example.

The source machine for Puppet Server - VPS Ubuntu 16.04 - 4Gb Memory, 2 CPU cores.

Go:
')
Important - the server and the managed node must be accessible to each other by name! If you don’t use DNS, you must register the hosts in / etc / hosts:
in our example on the server:

172.16.248.189 ubuntu

at the node:

172.16.248.34 puppet

Update the system on the server:

apt-get update
apt-get upgrade
apt-get install mc #

Add the official Puppet repository:

cd /opt
curl -O https://apt.puppetlabs.com/puppetlabs-release-pc1-xenial.deb
sudo dpkg -i puppetlabs-release-pc1-xenial.deb
sudo apt-get update

Install the package:

sudo apt-get install puppetserver

Configure memory allocation for Puppet Server (by default, Puppet uses 2Gb):

sudo mcedit /etc/default/puppetserver

JAVA_ARGS="-Xms3g -Xmx3g -XX:MaxPermSize=256m"

Start the server:

sudo systemctl start puppetserver

Make sure that everything is ok:

sudo systemctl status puppetserver

Add a server to autorun:

sudo systemctl enable puppetserver

Congratulations! This completes the installation and basic setup of the server!

Installing the puppet agent on a managed node:

Add the official Puppet repository:

cd /opt
curl -O https://apt.puppetlabs.com/puppetlabs-release-pc1-xenial.deb
sudo dpkg -i puppetlabs-release-pc1-xenial.deb
sudo apt-get update

Install the package:

sudo apt-get install puppet-agent

We start and add to autorun:

sudo systemctl start puppet
sudo systemctl enable puppet

If everything went well, we will see on the server a request to sign the certificate from the node, see the list of certificates on the server:

sudo /opt/puppetlabs/bin/puppet cert list --all

Sign certificate for node with uname ubuntu:

sudo /opt/puppetlabs/bin/puppet cert sign ubuntu

The result should be something like:

root@puppet:/var/log# sudo /opt/puppetlabs/bin/puppet cert list --all
+ "puppet.my-domain.org" (SHA256) A4:A8:4E:B0:81:7F:A0:84:F3:03:03:F0:DE:81:E8:73:A9:01:7A:90:F6:A2:27:0D:62:18:F9:D9:7B:F0:F0:9F (alt names: "DNS:puppet", "DNS:puppet.my-domain.org")
+ "ubuntu" (SHA256) 79:37:37:3F:D5:5C:C1:D3:FF:8D:BC:14:82:11:CE:9F:A6:4C:1C:90:3C:A6:A8:7D:E0:D8:81:D8:D7:D8:43:05

Let's test our installation - for example, create the default manifest puppet, the default path:

sudo nano /etc/puppetlabs/code/environments/production/manifests/site.pp
file {'/tmp/it_works.txt':
ensure => present,
mode => '0644',
content => "It works on ${ipaddress_eth0}!\n",
}

On a managed node, we “jerk” an agent — by forcing it to ask for configuration changes on the server:

sudo /opt/puppetlabs/bin/puppet agent --test

The result of the execution of this manifest should be the creation of the file “it_works.txt” on the node in
/ tmp /:

cat /tmp/it_works.txt
It works on 172.16.248.189!

Congratulations! You managed to apply the first Puppet manifest to a managed node!

Next, to speed up debugging and testing, we change the interval for the agent to contact the server (the interval is specified in seconds):

nano /etc/puppetlabs/puppet/puppet.conf
runinterval = 180

Now we will try to create and apply our first own module to the node in the default environment for nodes - pdoruction (bike) to install and configure the packages necessary for building logs from the node in Graylog (Graylog Collector Sidecar + Filebeat), on the server we will prepare the module directories:

mkdir /etc/puppetlabs/code/environments/production/collector_sidecar/modtest
mkdir /etc/puppetlabs/code/environments/production/modules/collector_sidecar/manifests
mkdir /etc/puppetlabs/code/environments/production/modules/collector_sidecar/files
chmod 777 /etc/puppetlabs/code/environments/production/modules/collector_sidecar/files
cd mkdir /etc/puppetlabs/code/environments/production/modules/collector_sidecar/manifests

Download the necessary installation packages to the / etc / puppetlabs / code / environments / production / modules / collector_sidecar / files directory:
collector-sidecar_0.1.4-1_amd64.deb
filebeat-5.6.4-amd64.deb
and the necessary configs in our case is the file:
collector_sidecar.yml

Create and edit the default manifest:

touch init.pp
nano init.pp

class collector_sidecar {
# collector-sidecar /opt puppet
file { '/opt/collector-sidecar_0.1.4-1_amd64.deb':
ensure => present,
mode => '0644',
source => 'puppet:///modules/collector_sidecar/collector-sidecar_0.1.4-1_amd64.deb',
}
# filebeat /opt puppet
file { '/opt/filebeat-5.6.4-amd64.deb':
ensure => present,
mode => '0644',
source => 'puppet:///modules/collector_sidecar/filebeat-5.6.4-amd64.deb',
}
# collector-sidecar
package { 'collector-sidecar':
provider => dpkg,
ensure => installed,
source => '/opt/collector-sidecar_0.1.4-1_amd64.deb',
}
# - collector_sidecar.yml etc/graylog/collector-sidecar/
file { '/etc/graylog/collector-sidecar/collector_sidecar.yml':
mode => '0644',
source => 'puppet:///modules/collector_sidecar/collector_sidecar.yml',
require => Package['collector-sidecar'] #, , - collector-sidecar
}
# filebeat
package { 'filebeat':
provider => dpkg,
ensure => installed,
source => '/opt/filebeat-5.6.4-amd64.deb',
require => Package['collector-sidecar']
}
# graylog-collector-sidecar
exec { 'install_gcs_service':
command => '/usr/bin/graylog-collector-sidecar -service install',
creates => '/etc/systemd/system/collector-sidecar.service',
require => Package['collector-sidecar']
}
# collector-sidecar
service { 'collector-sidecar':
ensure => running,
enable => true,
require => Package['collector-sidecar']
}
# filebeat
service { 'filebeat':
ensure => running,
enable => true,
require => Package['filebeat']
}
}

It is important to understand the philosophy behind Puppet — the manifests describe the “configuration state” of the system to which it should be brought.

Now we can “hang” our module in the manifest that is executed by default:

nano /etc/puppetlabs/code/environments/production/manifests/site.pp
#
node 'default' {
include 'collector_sidecar'
}

node 'ubuntu' {
include 'collector_sidecar'
}

To produce a dry-run manifest on the server locally - the launch will be emulated, but no actual actions will be performed:

/opt/puppetlabs/bin/puppet apply --noop /etc/puppetlabs/code/environments/production/manifests/site.pp

It is also convenient for debugging to watch the log on the Puppet server:


tail -f puppetserver.log

At the node, the puppet-agent writes a log to the syslog:
tail -f /var/log/syslog

Storing manifests in svn
We have a Subversion server, on the server we will prepare a repository for storing manifests:
creating repository:

sudo svnadmin create /media/datadrive/svn/puppet

Copy the manifests from the puppet server to the server with svn:

scp -r administrator@puppet.my-domainorg/etc/puppetlabs/code/ /home/administrator/code/

We import the manifests into the created repository:

svn import /home/administrator/code/ svn://svn.my-domain.org/puppet -m 'init'

Make a chek-out repository to the Pupppet manifest folder on the Puppet server:
if the svn client is not yet worth it:

apt install subversion
svn co svn://svn.my-domain.org/puppet /etc/puppetlabs/code --username=puppet

Now we create a working copy of the repository on our working machine and try to make changes to the manifest, commit, check on the Puppet server:

svn up /etc/puppetlabs/code/

If everything is OK, we add to the cron update of the working copy every nth amount of time. Or we can make a post-commit hook.