Centralized Configuration

Management

Puppet

The Foreman

Marionette Collective

Terms And Definitions

Agent

Puppetmaster

The server which stores, compiles, and distributes

the configuration manifests

Manifest

Any node which gets it's configuration from the

puppetmaster

The configuration stated in the puppet Domain

Specific Language

Catalog

The compiled manifest which is sent to the puppet

Puppet

Overview

Written in Ruby

Can be fault tolerant

Uses secure communications

Purposes

Deployment
Configuration
Abstraction Of Resources
Transactional Configurations

Puppet Components

Puppet Master

The puppetmaster server must be running a version

>= the latest puppet agent being managed
The FQDN of the puppet server will be used by all
puppet agents to access file/manifest/template
resources via the URL:

puppet://<puppetmaster>/(files|manifests|templates)

The puppetmaster maintains a filebucket which

archives the various configuration versions in case
they need to be reverted

Puppet Components

Puppet Agent

The agent runs on each managed host

It uses signed SSL certificates to perform server
and client authentication
It uploads the facts about the node and then
downloads the compiled manifest to be applied
Applies the manifest and then uploads the report of
what was changed, what remained the same, and
any errors which were encountered

Puppet Components

Puppet CA

The integrated certificate authority for puppet

Works automatically with the puppetmaster to
accept certificate requests, allow/deny clients, sign
certificate requests
Workflow:

When a new puppet agent is installed, the first run of the

agent will generate a new certificate request.
The certificate request will be sent, via SSL on port 8140,
to the puppetmaster
The puppetca will store the certificate request, but
manual signing of the request is required before the
agent can communicate with the puppetmaster

The Puppet Master Layout

/etc

/puppet

fileserver.conf
puppet.conf
auth.conf
/ssl
/manifests

site.pp

/templates
/files

Basics Of The Puppet

Langage

Types

File

Package

Cron

Exec

Service

Augeas

User

Group

ssh_authorized_key

Basics Of The Puppet

Langage

Node

Can contain puppet types and classes

Applies to a particular node

Class

A container which can wrap multiple puppet type

definitions and other classes

Parameterized Classes

Same as a class, but you can specify parameters to

change the behavior of the class

Basics Of The Puppet

Langage

Node Example
node "dub1.dns.com" {
package { openssh:
ensure
=> latest,
notify => Service['openssh'],
}
file { /etc/ssh/sshd_config:
ensure
=> present,
owner
=> root,
group
=> root,
mode
=> 0644,
source=> puppet://<puppetmaster>/files/etc/ssh/sshd_config,
require
=> Package['openssh'],
notify => Service['openssh'],
}
service { openssh:
ensure
=> running,
require
=> [Package['openssh'],File['/etc/ssh/sshd_config']],
hasstatus
=> true,
hasrestart
=> true,
}
}

Basics Of The Puppet

Langage

Class Example
class secureShell {
package { openssh:
ensure
=> latest,
notify => Service['openssh'],
}

file { /etc/ssh/sshd_config:
ensure
=> present,
owner
=> root,
group
=> root,
mode
=> 0644,
source=> puppet://<puppetmaster>/files/etc/ssh/sshd_config,
require
=> Package['openssh'],
notify => Service['openssh'],
}
service { openssh:
ensure
=> running,
require
=> [Package['openssh'],File['/etc/ssh/sshd_config']],
hasstatus
=> true,
hasrestart
=> true,
}
}

Basics Of The Puppet

Langage

Parameterized Class Example

class secureShell ( $sshPort=22, $useDNS=no, $passwordAuth=no, $enableSFTP ) {
package { openssh:
ensure
=> latest,
notify => Service['openssh'],
}

file { /etc/ssh/sshd_config:
ensure
=> present,
owner
=> root,
group
=> root,
mode
=> 0644,
content
=> template('sshd_config.erb'),
require
=> Package['openssh'],
notify => Service['openssh'],
}
service { openssh:
ensure
=> running,
require
=> [Package['openssh'],File['/etc/ssh/sshd_config']],
hasstatus
=> true,
hasrestart
=> true,
}
}

Basics Of The Puppet

Langage

SSH Template Example

Port <%= sshPort %>
Protocol 2
SyslogFacility AUTHPRIV
PasswordAuthentication <%= passwordAuth %>
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials yes
UsePAM yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL
X11Forwarding yes
UseDNS <%= useDNS %>
<% if enableSFTP==true -%>
Subsystem sftp /usr/libexec/openssh/sftp-server
<% end -%>

Using Facts In Puppet

facter command

Example facter output

architecture => x86_64
domain => dns.com
fqdn => mpdnsipv6.dns.com
hardwareisa => x86_64
hardwaremodel => x86_64
hostname => mpdnsipv6
interfaces => eth0,sit0
ipaddress => 91.212.12.99
is_virtual => true
kernel => Linux
kernelmajversion => 2.6
kernelrelease => 2.6.21-xen
kernelversion => 2.6.21
macaddress => 00:16:3E:4A:40:B6
memoryfree => 6.58 GB
memorysize => 8.00 GB
netmask => 255.255.255.192
operatingsystem => CentOS
operatingsystemrelease => 5.6

Using Facts In Puppet

A Class Using Facts

class openSSH ( $sshPort=22, $useDNS=no, $passwordAuth=no, $enableSFTP ) {

if ( $operatingsystem==redhat or $operatingsystem==RedHat ) {
package { openssh:
ensure
=> latest,
}
} else {
package { ssh:
ensure
=> latest,
alias
=> openssh,
}
}
file { /etc/ssh/sshd_config:
content
=> template(sshd_config.erb),
owner
=> root,
group
=> root,
mode
=> 0644,
ensure
=> present,
require
=> Package['openssh'],
}
}

Conditionals In Puppet
Manifests

The if/else statement

if ( $operatingsystem==redhat or $operatingsystem==RedHat ) {
package { openssh:
ensure
=> latest,
}
} else {
package { ssh:
ensure
=> latest,
alias
=> openssh,
}
}

The case statement

case $operatingsystem {
centos, redhat: { $apache = "httpd" }
debian, ubuntu: { $apache = "apache2" }
default: { fail("Unrecognized operating system for webserver") }
}

Conditionals In Puppet
Manifests

The selector

$apache = $operatingsystem ? {
centos
=> 'httpd',
redhat
=> 'httpd',
/(?i)(ubuntu|debian)/ => "apache2-$1",
# (Don't actually use that package name.)
default
=> undef,
}
package {$apache:
ensure
=> latest,
}

Package Management In
Puppet

The package type

Works with 31 different package management

systems
Allows the user to specify a particular version if
desired and the package manager supports it

You can override the default provider

package { ssh : ensure=>'5.5' }

Package { puppet : provider=>'gem' }

The ensure=>latest will make sure that the latest

package in the repository is always installed

Puppet Setup

Initial installation

Add the DNS.com repository to the yum/apt

configuration

YUM:
[comwired]

name=Comwired
baseurl=http://comwired:gr4n173@opennms.dns.com/repo/centos5/5/

enabled=1
gpgcheck=0

APT:

deb http://comwired:gr4n173@opennms.dns.com/apt maverick main

Puppet Setup

For debian/ubuntu hosts, add the apt key

wget -O- -q http://opennms.dns.com/apt-repo.pub |

apt-key add

Install puppet

aptitude update && aptitude install puppet

yum install puppet

Puppet Setup

Configure puppet

Edit /etc/puppet/puppet.conf

Add the following settings:

Generate a certificate request

server = lou1
certname = <fqdn>

puppet agent --test

Sign the certificate request from the

puppetmaster

puppetca -s <fqdn>

Puppet Management

Revoking a certificate

Revoke and clean a certificate

puppetca -c <fqdn>

List all active certificates

puppetca -r <fqdn>

puppetca -l -a

NOTE: When using Apache/Passenger to run

the puppetmaster, you must restart apache for
a certificate revokation to take effect

Puppet Management

Version Control

Makes puppet usable by multiple people

Makes reverting changes more reliable

Allows for change control reporting and history

Puppet Management

Gepetto

Based on the Eclipse platform

Provides a single editor/version control/validation
interface for puppet

The Foreman

The Foreman

Provides Visualization Of Puppet Activity

View Facts

View Reports

Manage Provisioning Of Virtual Machines

The Foreman

Installation

http://theforeman.org/projects/foreman/wiki/Installati
on_instructions

Marionette Collective

What is mcollective?

Interact with small to very large clusters of servers

Use a broadcast paradigm for request distribution.
All servers get all requests at the same time,
requests have filters attached and only servers
matching the filter will act on requests. There is no
central asset database to go out of sync, the
network is the only source of truth.
Break free from ever more complex naming
conventions for hostnames as a means of identity.
Use a very rich set of meta data provided by each
machine to address them. Meta data comes from
Puppet, Chef, Facter, Ohai or plugins you provide

Marionette Collective

Installation

A Stomp server (Like ActiveMQ)

Ruby

Rubygems

Ruby Stomp Client (Version >=1.1.8 recommended)

Marionette Collective

Installation (cont.)

DNS.com has already created ready-to-use

packages for all of these items
DNS.com has already created puppet manifests for
installing and configuring mcollective

Marionette Collective

Usage

From a host which has the mcollective-client and

the mcollective-plugins packages installed and
configured

root@lou1:~# mco ping

lou1
time=46.91 ms
infra
time=84.60 ms
chi1.dns.com
time=85.09 ms
ip-10-194-58-223.ec2.internal
time=100.46 ms
dal2.dns.com
time=109.47 ms
t8eddnrecur2
time=127.74 ms
t8eddnipcache1
time=128.28 ms
t8eddnrecur1
time=128.80 ms
atl1.dns.com
time=132.18 ms
sea1.dns.com
time=169.24 ms

---- ping statistics ---28 replies max: 554.15 min: 46.91 avg: 221.43

Marionette Collective
root@lou1:~# mco inventory chi1.dns.com
Inventory for chi1.dns.com:
Server Statistics:
Version: 1.2.1
Start Time: Tue Sep 27 08:31:00 -0400 2011
Config File: /etc/mcollective/server.cfg
Collectives: mcollective
Main Collective: mcollective
Process ID: 27281
Total Messages: 27435
Messages Passed Filters: 14485
Messages Filtered: 12950
Replies Sent: 14484
Total Processor Time: 47.21 seconds
System Time: 8.29 seconds
Agents:
discovery
filemgr
iptables
nettest
nrpe
puppet-package
puppet-service puppetca
puppetd
puppetral
rpcutil
Configuration Management Classes:
basenode
chi1.dns.com
comwiredrepo
dnsfirewall
dnsinterface
dnsnode
geodns
hosts
mcollective
ntp
resolver
settings
snmpconfig
useraccounts

Facts:
mcollective => 1

Marionette Collective

Installation And Configuration

Handled By Puppet Manifest

Packaged By DNS.com

Marionette Collective

Just The Facts . . .

mco facts -v processorcount

root@home:/home/dphillips# mco facts -v processorcount
Determining the amount of hosts matching filter for 2 seconds .... 33
Report for fact: processorcount
1

found 2 times
awsstub1
awsstub2

16

found 1 times
chi1.dns.com

mco facts -v ipaddress

root@home:/home/dphillips# mco facts -v ipaddress
Determining the amount of hosts matching filter for 2 seconds .... 33
Report for fact: ipaddress
10.194.58.223

found 1 times

ip-10-194-58-223.ec2.internal
10.2.32.13
h6eddnrecur2

found 1 times

Marionette Collective

Managing Puppet From mcollective

mco puppetd runonce

mco puppetd runall <concurrency>

mco puppetd runonce -I <identity>

mco puppetd runonce -W /<regex>/

mco puppetd runonce -C <class name>

mco puppetd runonce -F <fact>=<value>

mco puppetd status

Marionette Collective

Just The Facts (cont.) . . .

Setting Facts For Identification Purposes

Setting Facts For Inventory Purposes

Setting Facts For Tracking Purposes

Setting Facts For Versioning

Marionette Collective

To use puppet with Marionette Collective, you

must disable puppet running as a service

The puppet service uses the same lock file as the

puppet agent runonce command and thus if it is
running as a service, the mcollective runs will fail.

Instead, use a cron job to run puppet jobs via

mcollective

mco puppetd runall <concurrency>

The concurrency argument determines the maximum

number of puppet agents running at a given time to
ensure that the puppetmaster does not get overwhelmed

Questions And Answers

What if I need to make a change outside of

puppet/mcollective?
OK, I'm still going to have to make a change
outside of puppet, how can I accomplish it
without having puppet/mcollective change it
back?
Do I have to use puppet to do everything?
Seems like this adds a lot of hassle, what are
the gains?

Resources

http://puppetlabs.com/
Pro Puppet By James Turnbull and Jeffrey
McCune

IRC: freenode.org - #puppet/#mcollective/

#theforeman