Centralized Configuration

Management
Puppet
The Foreman
Marionette Collective
Terms And Definitions
Agent
Any node which gets it's configuration from the
puppetmaster
Puppetmaster
The server which stores, compiles, and distributes
the configuration manifests
Manifest
The configuration stated in the puppet Domain
Specific Language
Catalog
The compiled manifest which is sent to the puppet
agent. It can contain configuration rules and/or
compiled templates
Puppet
Overview
Written in Ruby
Can be fault tolerant
Uses secure communications
Purposes
Deployment
Configuration
Abstraction Of Resources
Transactional Configurations
Puppet Components
Puppet Master
The puppetmaster server must be running a version
>= the latest puppet agent being managed
The FQDN of the puppet server will be used by all
puppet agents to access file/manifest/template
resources via the URL:
puppet://<puppetmaster>/(files|manifests|templates)
The puppetmaster maintains a filebucket which
archives the various configuration versions in case
they need to be reverted
Puppet Components
Puppet Agent
The agent runs on each managed host
It uses signed SSL certificates to perform server
and client authentication
It uploads the facts about the node and then
downloads the compiled manifest to be applied
Applies the manifest and then uploads the report of
what was changed, what remained the same, and
any errors which were encountered
Puppet Components
Puppet CA
The integrated certificate authority for puppet
Works automatically with the puppetmaster to
accept certificate requests, allow/deny clients, sign
certificate requests
Workflow:
When a new puppet agent is installed, the first run of the
agent will generate a new certificate request.
The certificate request will be sent, via SSL on port 8140,
to the puppetmaster
The puppetca will store the certificate request, but
manual signing of the request is required before the
agent can communicate with the puppetmaster
The Puppet Master Layout
/etc
/puppet
fileserver.conf
puppet.conf
auth.conf
/ssl
/manifests
site.pp
/templates
/files
Basics Of The Puppet
Langage
Types
File
Package
Cron
Exec
Service
Augeas
User
Group
ssh_authorized_key
Basics Of The Puppet
Langage
Node
Can contain puppet types and classes
Applies to a particular node
Class
A container which can wrap multiple puppet type
definitions and other classes
Parameterized Classes
Same as a class, but you can specify parameters to
change the behavior of the class
Basics Of The Puppet
Langage
Node Example
node "dub1.dns.com" {

package { openssh:
ensure => latest,
notify => Service['openssh'],
}

file { /etc/ssh/sshd_config:
ensure => present,
owner => root,
group => root,
mode => 0644,
source => puppet://<puppetmaster>/files/etc/ssh/sshd_config,
require => Package['openssh'],
notify => Service['openssh'],
}

service { openssh:
ensure => running,
require => [Package['openssh'],File['/etc/ssh/sshd_config']],
hasstatus => true,
hasrestart => true,
}
}
Basics Of The Puppet
Langage
Class Example
class secureShell {
package { openssh:
ensure => latest,
notify => Service['openssh'],
}

file { /etc/ssh/sshd_config:
ensure => present,
owner => root,
group => root,
mode => 0644,
source => puppet://<puppetmaster>/files/etc/ssh/sshd_config,
require => Package['openssh'],
notify => Service['openssh'],
}

service { openssh:
ensure => running,
require => [Package['openssh'],File['/etc/ssh/sshd_config']],
hasstatus => true,
hasrestart => true,
}
}
Basics Of The Puppet
Langage
Parameterized Class Example
class secureShell ( $sshPort=22, $useDNS=no, $passwordAuth=no, $enableSFTP ) {
package { openssh:
ensure => latest,
notify => Service['openssh'],
}

file { /etc/ssh/sshd_config:
ensure => present,
owner => root,
group => root,
mode => 0644,
content => template('sshd_config.erb'),
require => Package['openssh'],
notify => Service['openssh'],
}

service { openssh:
ensure => running,
require => [Package['openssh'],File['/etc/ssh/sshd_config']],
hasstatus => true,
hasrestart => true,
}
}
Basics Of The Puppet
Langage
SSH Template Example

Port <%= sshPort %>
Protocol 2
SyslogFacility AUTHPRIV
PasswordAuthentication <%= passwordAuth %>
ChallengeResponseAuthentication no
GSSAPIAuthentication yes
GSSAPICleanupCredentials yes
UsePAM yes
AcceptEnv LANG LC_CTYPE LC_NUMERIC LC_TIME LC_COLLATE LC_MONETARY LC_MESSAGES
AcceptEnv LC_PAPER LC_NAME LC_ADDRESS LC_TELEPHONE LC_MEASUREMENT
AcceptEnv LC_IDENTIFICATION LC_ALL
X11Forwarding yes
UseDNS <%= useDNS %>
<% if enableSFTP==true -%>
Subsystem sftp /usr/libexec/openssh/sftp-server
<% end -%>
Using Facts In Puppet
facter command
Example facter output
architecture => x86_64
domain => dns.com
fqdn => mpdnsipv6.dns.com
hardwareisa => x86_64
hardwaremodel => x86_64
hostname => mpdnsipv6
interfaces => eth0,sit0
ipaddress => 91.212.12.99
is_virtual => true
kernel => Linux
kernelmajversion => 2.6
kernelrelease => 2.6.21-xen
kernelversion => 2.6.21
macaddress => 00:16:3E:4A:40:B6
memoryfree => 6.58 GB
memorysize => 8.00 GB
netmask => 255.255.255.192
operatingsystem => CentOS
operatingsystemrelease => 5.6
Using Facts In Puppet
A Class Using Facts
class openSSH ( $sshPort=22, $useDNS=no, $passwordAuth=no, $enableSFTP ) {
if ( $operatingsystem==redhat or $operatingsystem==RedHat ) {
package { openssh:
ensure => latest,
}
} else {
package { ssh:
ensure => latest,
alias => openssh,
}
}

file { /etc/ssh/sshd_config:
content => template(sshd_config.erb),
owner => root,
group => root,
mode => 0644,
ensure => present,
require => Package['openssh'],
}
}
Conditionals In Puppet
Manifests
The if/else statement




The case statement

if ( $operatingsystem==redhat or $operatingsystem==RedHat ) {
package { openssh:
ensure => latest,
}
} else {
package { ssh:
ensure => latest,
alias => openssh,
}
}

case $operatingsystem {
centos, redhat: { $apache = "httpd" }
debian, ubuntu: { $apache = "apache2" }
default: { fail("Unrecognized operating system for webserver") }
}

Conditionals In Puppet
Manifests
The selector
$apache = $operatingsystem ? {
centos => 'httpd',
redhat => 'httpd',
/(?i)(ubuntu|debian)/ => "apache2-$1",
# (Don't actually use that package name.)
default => undef,
}

package {$apache:
ensure => latest,
}

Package Management In
Puppet
The package type
Works with 31 different package management
systems
Allows the user to specify a particular version if
desired and the package manager supports it
package { ssh : ensure=>'5.5' }
You can override the default provider
Package { puppet : provider=>'gem' }
The ensure=>latest will make sure that the latest
package in the repository is always installed
Puppet Setup
Initial installation
Add the DNS.com repository to the yum/apt
configuration
YUM:
[comwired]
name=Comwired
baseurl=http://comwired:gr4n173@opennms.dns.com/repo/centos5/5/
enabled=1
gpgcheck=0
APT:
deb http://comwired:gr4n173@opennms.dns.com/apt maverick main
Puppet Setup
For debian/ubuntu hosts, add the apt key
wget -O- -q http://opennms.dns.com/apt-repo.pub |
apt-key add
Install puppet
aptitude update && aptitude install puppet
yum install puppet
Puppet Setup
Configure puppet
Edit /etc/puppet/puppet.conf
Add the following settings:
server = lou1
certname = <fqdn>
Generate a certificate request
puppet agent --test
Sign the certificate request from the
puppetmaster
puppetca -s <fqdn>
Puppet Management
Revoking a certificate
puppetca -r <fqdn>
Revoke and clean a certificate
puppetca -c <fqdn>
List all active certificates
puppetca -l -a
NOTE: When using Apache/Passenger to run
the puppetmaster, you must restart apache for
a certificate revokation to take effect
Puppet Management
Version Control
Makes puppet usable by multiple people
Makes reverting changes more reliable
Allows for change control reporting and history
Puppet Management
Gepetto
Based on the Eclipse platform
Provides a single editor/version control/validation
interface for puppet
The Foreman
The Foreman
Provides Visualization Of Puppet Activity
View Facts
View Reports
Manage Provisioning Of Virtual Machines
The Foreman
Installation
http://theforeman.org/projects/foreman/wiki/Installati
on_instructions
Marionette Collective
What is mcollective?
Interact with small to very large clusters of servers
Use a broadcast paradigm for request distribution.
All servers get all requests at the same time,
requests have filters attached and only servers
matching the filter will act on requests. There is no
central asset database to go out of sync, the
network is the only source of truth.
Break free from ever more complex naming
conventions for hostnames as a means of identity.
Use a very rich set of meta data provided by each
machine to address them. Meta data comes from
Puppet, Chef, Facter, Ohai or plugins you provide
yourself.
Comes with simple to use command line tools to
call remote agents.
Ability to write custom reports about your
infrastructure.
A number of agents to manage packages, services
and other common components are available from
the community.
Allows you to write simple RPC style agents, clients
and Web UIs in an easy to understand language -
Ruby
Extremely pluggable and adaptable to local needs
Middleware systems already have rich
authentication and authorization models, leverage
these as a first line of control. Include fine grained
Authentication using SSL or RSA, Authorization and
Auditing of requests. You can see more details in
the Security Overview.
Re-use the ability of middleware to do clustering,
routing and network isolation to realize secure and
scalable setups.
Marionette Collective
Installation
A Stomp server (Like ActiveMQ)
Ruby
Rubygems
Ruby Stomp Client (Version >=1.1.8 recommended)
Marionette Collective
Installation (cont.)
DNS.com has already created ready-to-use
packages for all of these items
DNS.com has already created puppet manifests for
installing and configuring mcollective
Marionette Collective
Usage
From a host which has the mcollective-client and
the mcollective-plugins packages installed and
configured
root@lou1:~# mco ping
lou1 time=46.91 ms
infra time=84.60 ms
chi1.dns.com time=85.09 ms
ip-10-194-58-223.ec2.internal time=100.46 ms
dal2.dns.com time=109.47 ms
t8eddnrecur2 time=127.74 ms
t8eddnipcache1 time=128.28 ms
t8eddnrecur1 time=128.80 ms
atl1.dns.com time=132.18 ms
sea1.dns.com time=169.24 ms


---- ping statistics ----
28 replies max: 554.15 min: 46.91 avg: 221.43


Marionette Collective
root@lou1:~# mco inventory chi1.dns.com
Inventory for chi1.dns.com:

Server Statistics:
Version: 1.2.1
Start Time: Tue Sep 27 08:31:00 -0400 2011
Config File: /etc/mcollective/server.cfg
Collectives: mcollective
Main Collective: mcollective
Process ID: 27281
Total Messages: 27435
Messages Passed Filters: 14485
Messages Filtered: 12950
Replies Sent: 14484
Total Processor Time: 47.21 seconds
System Time: 8.29 seconds

Agents:
discovery filemgr iptables
nettest nrpe puppet-package
puppet-service puppetca puppetd
puppetral rpcutil

Configuration Management Classes:
basenode chi1.dns.com
comwiredrepo dnsfirewall
dnsinterface dnsnode
geodns hosts
mcollective ntp
resolver settings
snmpconfig useraccounts


Facts:
mcollective => 1

Marionette Collective
Installation And Configuration
Handled By Puppet Manifest
Packaged By DNS.com
Marionette Collective
Just The Facts . . .
mco facts -v processorcount




mco facts -v ipaddress
root@home:/home/dphillips# mco facts -v processorcount
Determining the amount of hosts matching filter for 2 seconds .... 33
Report for fact: processorcount

1 found 2 times

awsstub1
awsstub2

16 found 1 times

chi1.dns.com
root@home:/home/dphillips# mco facts -v ipaddress
Determining the amount of hosts matching filter for 2 seconds .... 33
Report for fact: ipaddress

10.194.58.223 found 1 times

ip-10-194-58-223.ec2.internal

10.2.32.13 found 1 times

h6eddnrecur2
Marionette Collective
Managing Puppet From mcollective
mco puppetd runonce
mco puppetd runall <concurrency>
mco puppetd runonce -I <identity>
mco puppetd runonce -W /<regex>/
mco puppetd runonce -C <class name>
mco puppetd runonce -F <fact>=<value>
mco puppetd status
Marionette Collective
Just The Facts (cont.) . . .
Setting Facts For Identification Purposes
Setting Facts For Inventory Purposes
Setting Facts For Tracking Purposes
Setting Facts For Versioning
Marionette Collective
To use puppet with Marionette Collective, you
must disable puppet running as a service
The puppet service uses the same lock file as the
puppet agent runonce command and thus if it is
running as a service, the mcollective runs will fail.
Instead, use a cron job to run puppet jobs via
mcollective
mco puppetd runall <concurrency>
The concurrency argument determines the maximum
number of puppet agents running at a given time to
ensure that the puppetmaster does not get overwhelmed
Questions And Answers
What if I need to make a change outside of
puppet/mcollective?
OK, I'm still going to have to make a change
outside of puppet, how can I accomplish it
without having puppet/mcollective change it
back?
Do I have to use puppet to do everything?
Seems like this adds a lot of hassle, what are
the gains?
Resources
http://puppetlabs.com/
Pro Puppet By James Turnbull and Jeffrey
McCune
IRC: freenode.org - #puppet/#mcollective/
#theforeman