Tuesday 19 February 2019

Debugging with the puppet agent command

To enable debugging on a puppet agent you can issue:

puppet agent --test --noop --debug --trace

The above command does not actually carry out any changes - it only simulates what could have happened. Removing the '--noop' switch allows us to actually carry out those changes e.g.:

puppet agent --test --debug --trace

Tuesday 5 February 2019

Creating your first Puppet module

This tutorial will be a quick demonstration of how to create, build and test a basic puppet module.

Firstly generate the puppet module with:

mkdir ~/workbench && cd ~/workbench

puppet module generate username-webmin --skip-interview

This will generate the following directory structure:

.
└── webmin (module directory)
    ├── examples
    │   └── init.pp (example of how to initailize the class)
    ├── Gemfile (used to describe dependancies needed for the module / Ruby)
    ├── manifests (holds the module manifests i.e. contains a set of instructions that need to be run)
    │   └── init.pp (the default manifest - it defines our main class: webmin)
    ├── metadata.json (contains module metadata like author, module description, dependancies etc.)
    ├── Rakefile (essentially a makefile for Ruby)
    ├── README.md (contains module documentation)
    └── spec (used for automated testing - is optional)
        ├── classes
        │   └── init_spec.rb
        └── spec_helper.rb

If we do a cat on the init.pp within the examples directory:

cat examples/init.pp

include ::webmin

This include statement can be used in other manifests in Puppet and simply imports the webmin class.

In older modules you might have seen a params manifest (params.pp) - this design pattern has recently been replaced with the release of Hiera v5 with in-module data (https://github.com/puppetlabs/best-practices/blob/master/puppet-module-design.md)

This means that we need to construct our Hiera hierarchy in the form of 'hiera.yaml' in our module. This will typically look like this:

cat webmin/hiera.yaml

---
version: 5

defaults:
  datadir: 'data'
  data_hash: 'yaml_data'

hierarchy:
  - name: 'Full Version'
    path: '%{facts.os.name}-%{facts.os.release.full}.yaml'

  - name: 'Major Version'
    path: '%{facts.os.name}-%{facts.os.release.major}.yaml'

  - name: 'Distribution Name'
    path: '%{facts.os.name}.yaml'

  - name: 'Operating System Family'
    path: '%{facts.os.family}-family.yaml'

  - name: 'common'
    path: 'common.yaml'

Note: It's important to keep in mind that the hierarchy is reusable e.g. avoid adding in specific networks or environments.

We'll also need to create our data directory to hold our yaml data in:

mkdir webmin/data
touch webmin/data/Debian-family.yaml
touch webmin/data/RedHat-family.yaml
touch webmin/data/common.yaml

For the sake of time and simplicity I have confined the yaml data to a few OS families.

As seen in the hierarchy anything defined within the 'Operating System Family' take presidense over anything in 'common'. However if this module were to be applied to say OpenSUSE only settings in common.yaml would be applied - so it's important to ensure that there are suitable defaults in common data.

We'll proceed by defining our variables within the init.pp (manifests directory):

class webmin (
  Boolean $install,
  Optional[Array[String]] $users,
  Optional[Integer[0, 65535]] $portnum,
  Optional[Stdlib::Absolutepath] $certificate,
  )
  {
  notify { 'Applying class webmin...': }
  }

In the above class we are defining several variables that will allow the user to customise how the module configures webmin. As you'll notice three of them are optional - so if they are undefined we'll provide the defaults our self in the manifest.

At this point we might want to verify the syntax in our init.pp manifest is valid - we can do this with:

puppet parser validate webmin/manifests/init.pp

We will also create two more classes - one for setup / installation of webmin and the other for configuration of it:

touch /webmin/manifests/install.pp && touch /webmin/manifests/configure.pp

We'll declare these in our init.pp like follows:

class webmin (
  Boolean $install,
  String $webmin_package_name,
  Optional[Array[String]] $users,
  Optional[Integer[0, 65535]] $portnum,
  Optional[Stdlib::Absolutepath] $certificate,
  )
  {
  notify { 'Applying webmin class...': }

  contain webmin::install
  contain webmin::configure

  Class['::webmin::install']
  -> Class['::webmin::configure']

  }

And then declare the classes:

cat /webmin/manifests/install.pp

# @summary
#   This class handles the webmin package.
#
# @api private
#
class webmin::install {

  if $webmin::install {

    package { $webmin::webmin_package_name:
      ensure => present,
    }

    service { $webmin::webmin_package_name:
    ensure    => running,
    enable    => true,
    subscribe => Package[$webmin::webmin_package_name],
    }

  }

}

and

cat /webmin/manifests/configure.pp

# @summary
#   This class handles webmin configuration.
#
# @api private
#
class webmin::configure {

  }

}

You'll notice that although we have defined variables in the parent class we have not actually defined what these will be yet. This is where Hiera will come in - so to data/common.yaml we add:

---
webmin::webmin_package_name: webmin
webmin::install: true
webmin::users: ~
webmin::portnum: 10000
webmin::certificate: ~

We'll also need to add repositories for webmin - as by default (to my knowledge at least) it's not included in any of the base repositories included with Redhat or Debian.

In order to configure repositories for particular operating systems we will need to use the yum and apt modules. Module dependencies are defined in the 'metadata.json' file - for example:

"dependencies": [
  { "name": "puppet/yum", "version_requirement": ">= 3.1.0" },
  { "name": "puppetlabs/apt", "version_requirement": ">= 6.1.0" },
  { "name": "puppetlabs/stdlib", "version_requirement": ">= 1.0.0" }
],

So your metadata.json would look something like:

{
  "name": "username-webmin",
  "version": "0.1.0",
  "author": "Joe Bloggs",
  "summary": "A module used to perform installation and configuration of webmin.",
  "license": "Apache-2.0",
  "source": "https://yoursite.com/puppetmodule",
  "project_page": "https://yoursite.com/puppetmodule",
  "issues_url": "https://yoursite.com/puppetmodule/issues",
  "dependencies": [
  { "name": "puppet/yum", "version_requirement": ">= 3.1.0" },
  { "name": "puppetlabs/apt", "version_requirement": ">= 6.1.0" },
  { "name": "puppetlabs/stdlib", "version_requirement": ">= 1.0.0" }
  ],
  "data_provider": null
}

Because we haven't packaged up the module yet we'll need to install the dependencies manually i.e.:

puppet module install puppetlabs/apt
puppet module install puppet/yum

We'll then need to copy them to our working directory (~/workbench) e.g.:

cp -R /etc/puppetlabs/code/environments/production/modules/{yum,apt,stdlib} ~/workbench

We'll use Hiera to configure the OS specific settings for the repositories:

cat webmin/data/RedHat-family.yaml

---
version: 5

yum::managed_repos:
    - 'webmin_repo'

yum::repos:
    webmin_repo:
        ensure: 'present'
        enabled: true
        descr: 'Webmin Software Repository'
        baseurl: 'https://download.webmin.com/download/yum'
        mirrorlist: 'https://download.webmin.com/download/yum/mirrorlist'
        gpgcheck: true
        gpgkey: 'http://www.webmin.com/jcameron-key.asc'
        target: '/etc/yum.repos.d/webmin.repo'

cat webmin/data/Debian-family.yaml

---
version: 5

apt::source
    webmin_repo
        location: 'http://download.webmin.com/download/repository'
        release: 'stretch'
        repos: 'contrib'
        key: '1B24BE83'
        key_source: 'http://www.webmin.com/jcameron-key.asc'
        include_src: 'false'

Let's check everything looks good:

puppet parser validate manifests/*.pp

We'll now try and apply our manifests and see if they apply correctly:

puppet apply --modulepath=~/workbench webmin/tests/init.pp

With any luck you will now have webmin installed and should also see our notify messages we included earlier.

Finally we can publish the module with:

sudo puppet module build webmin-reloaded

We can now distribute this internally or alternatively upload it for public consumption at the Puppet Forge.

To test the built module you can run:

sudo puppet module install ~/Workbench/username-webmin-0.1.0.tar.gz