Menu
Review: Puppet Enterprise 3.0 pulls more strings

Review: Puppet Enterprise 3.0 pulls more strings

Version 3.0 of Puppet Labs' configuration automation tool shines with speed boosts, orchestration improvements, and deeper support for Windows servers

With virtualization driving the meteoric rise in server instance counts, it's no wonder that so many configuration automation tools are hitting their stride. Where once we either did without configuration management or created our own custom scripts, we now have many options to manage changes to large numbers of servers.

One of the best options is Puppet Enterprise, the commercial version of the Puppet open source project. Version 3.0, just arrived, passes along many advances borne from the community edition, as well as new features found only in the enterprise version.

[ Puppet or Chef: The configuration management dilemma | 12 clever cloud tools for devs and ops | Subscribe to InfoWorld's Data Center newsletter to stay on top of the latest developments. ]

If you're already well versed in Puppet, then this version will bring with it welcome changes. The first is the speed. Although I wasn't able to fully corroborate Puppet Labs' claims of a 60 percent faster catalog core compilation, 2.7x speed in agent runtimes, or the heralded "200 percent performance boost," the reality is definitely close to that. For longtime Puppet users with large-scale deployments, the speed improvements are a very big deal, as they will reduce the load on both Puppet servers (that is, Masters) and managed nodes, as well as the time required to roll out changes across the board.

Puppet Enterprise 3.0 also introduces a new orchestration engine that allows for more granular and more stable large-scale changes. As compared to earlier versions, the new orchestration features include finer-grained filtering to select subsets of systems to modify or directly control, as well as batching functions to permit a gradual rollout of changes rather than performing them all at once.

Another major change is the deeper support of Windows servers. Previously, many actions such as Live Management would not work with Windows. The new version brings Live Management (a Web UI for managing nodes in real time) and orchestration of Windows servers into the fold.

Puppet Enterprise supports a broad range of platforms, providing Red Hat, Ubuntu, Debian, Solaris, and AIX packages, as well as a general Unix tarball. In addition, the open source community has released code for a variety of other operating systems. These packages are unsupported by Puppet Labs, but they offer some functionality for other Linux distributions, various BSD flavors, and Mac OS X.

Inside the PuppetPuppet is definitely a Linux/Unix-centric app. Although there is greater support for Windows in the new version, this tool is designed by and for Linux/Unix admins. Any Puppet deployment larger than a smattering of servers will require custom code and a fairly deep knowledge of Linux/Unix internals. This is not a click-and-drag tool, nor a Web-based configuration management tool, but a configuration file-based tool with a Web UI for monitoring and organizational tasks -- as it should be.

Working with Puppet requires learning Puppet's configuration language and knowing your way around Puppet's classes and modules, which are organized in configuration files in a specific hierarchy under /etc/puppetlabs/puppet. Without going into the heavy details, you can consider all modules to be includes in a big script, with classes defined and linked to specific server nodes or groups of nodes. This provides a very straightforward method of dealing with the complex and detailed requirements of managing multiple server platforms and distributions, not to mention switching and storage applications.

The Web UI included with Puppet Enterprise is well appointed and straightforward, allowing for the creation and management of nodes, groups, and classes, as well as report generation and viewing. It is a useful tool for manipulating node organization and quickly determining issues with certain parts of the framework. It also provides limited Live Management of nodes or groups of nodes. However, for many implementations it may be easier to think of the UI as read-only, and to work mostly through the command line. The deeper into Puppet you get, the more likely this will become, as you may discover that you can only apply certain configurations and modules to nodes and groups through the CLI, and these changes may not be reflected in the Web UI.

An example might be a module downloaded through the Puppet Forge module-sharing site that cannot use a class due to the nature of what it's configuring. In this case, you will need to apply the module to nodes or groups from within the Puppet configuration files, as you can't define a class that maps to a class definition within the Web UI. This configuration element will thus not appear in the Web UI.

I also encountered a few bugs in the Web UI, such as the fact that you can add a group with a name containing a period, but you cannot thereafter edit or delete that group through the UI. This would seem to be an input validation bug, because it appears that a period is an illegal character in group names.

Standing up the PuppetInstalling Puppet is conceptually very simple. Choose a supported platform for your Puppet Master server, and run the install script. This will prompt for a variety of packages and options to be installed: the Puppet Master, PostgreSQL, Cloud Provisioner, and so forth. The install script will also prompt for a username and password for the first administrator in the Web UI.

It's all very simple, but somewhat lacking in error checking and error reporting. For instance, if you install on a Linux system that has PostgreSQL already installed but not configured or running, the installer will happily install its own PostgreSQL binaries -- then try to use the wrong client to connect, which will fail, and produce a fairly useless error message saying that the PostgreSQL server failed to start, even though the server was clearly running. Fortunately after some digging, that root cause was discovered and fixed, but it wasn't the only time the installer bailed without any useful logging or apparent cause. The installation scripts are simple bash scripts, so debugging wasn't a big challenge.

Puppet Enterprise's main console display shows a list of nodes and the collective run status of all nodes.

These quibbles and pitfalls aside, the initial configuration is very straightforward, requiring DNS for all nodes and (obviously) clear network connectivity. Once the Puppet Master is installed, each node is configured essentially the same way, by running an installer for that node's platform, which generates a certificate and contacts the master server for approval. When the node is approved, it becomes configurable through Puppet and can be added to groups or bound to classes through the Web UI or CLI.

Then comes the harder part: configuring Puppet to make changes to those systems. This is done through modules, either custom-coded or downloaded from the Puppet Forge site and customized.

Puppet in actionAs a very simple example of how Puppet works, let's look at how we would use Puppet to make sure that the NTP (Network Time Protocol) service is properly configured and running on a selection of nodes.

Assuming we already have the nodes approved, we need to download a module, such as the third-party erwbgy-ntp module, written by Keith Burdis. We do this by installing the module from the command line:

puppet module install erwbgy-ntp

This downloads the module and places it under /etc/puppetlabs/puppet/modules/ntp. We then take a look at the file /etc/puppetlabs/puppet/modules/ntp/manifests/init.pp. It contains a number of configuration options for the module. For this example, we only need to modify the NTP server we want to use, so we modify the $servers variable:

$servers = [ '192.168.32.10', '192.168.16.10' ],

This will cause the module to add those two servers to the ntp.conf file. The init.pp file includes the services.pp file, which reads thus:

class ntp::service {

  service { 'ntpd':

    ensure     => running,

    hasstatus  => true,

    hasrestart => true,

    enable     => true,

    require    => Class['ntp::config'],

  }

}

This defines a subclass to handle the ntpd service itself. These variables control whether or not Puppet ensures that the service is running and enabled.

Another included file called install.pp handles package installations:

class ntp::install {

  case $::operatingsystem {

    'RedHat', 'CentOS', 'OracleLinux': {

      if ! defined(Package['ntp']) {

        package { 'ntp':  ensure => installed }

      }

      if versioncmp($::operatingsystemrelease, '6.0') > 0 {

        if ! defined(Package['ntpdate']) {

          package { 'ntpdate':  ensure => installed }

        }

      }

    }

    default: {

      fail('Currently only works on RedHat-like systems')

    }

  }

}

This code checks to make sure it's running on a compatible distribution (Red Hat, CentOS, or Oracle Linux in this case), and if it is, will cause the ntp package to be installed. It will also check to make sure that ntpdate is installed if required. Otherwise, an error will be thrown.

Taken all together, when applied to a target node, this module will install the ntp package if it's not already installed, will add lines to the configuration file to define NTP servers as 192.168.32.10 and 192.168.16.10, and will start the ntpd service if it's not already running.

The detail page of a specific Ubuntu server lists the groups and classes it belongs to and includes graphs of the agent runtimes and status for the past 30 days.

To apply this module to a node, we would then define a new class in the Web UI called ntp, edit a node group or a node itself, and add that class to the group or node. This will cause Puppet to apply that configuration to the node through the Puppet agent installed on the node.

By default, the agent checks in with the Puppet Master every 30 minutes, but of course that is a configurable parameter. The node can also be forced to update through the Live Management tab in the Web UI. Live Management allows an admin to enable, disable, run, and monitor the Puppet agent on a node or group of nodes. We can select a node to refresh, then force the refresh. If we select the node we configured with our ntp module, it will check in with the server, download the module information, install the ntp package, modify the configuration file, and start the service.

If somewhere down the line, the node checks in with the master and discovers the configuration has been manually changed or the service has been stopped or uninstalled, the agent will again apply the changes and start the service back up.

This is a very simple example of what Puppet can do. There are hundreds of modules available from Puppet Labs and contributors that can be used to manage a wide variety of configuration elements on Linux and Windows systems, across a wide variety of distributions and versions.

If we were to do the same thing for Windows servers, we might use the adenning/winntp module by Adam Denning. This is configured similarly on the Puppet Master:

class winntp (

  $special_poll_interval    = 900, # 15 minutes

  $ntp_server               = 192.168.32.10, 192.168.16.10',

  $max_pos_phase_correction = 54000, # 15 hours

  $max_neg_phase_correction = 54000, # 15 hours {

  include 'registry'

  service { 'w32time':

    ensure => 'running',

  }

...

)

When applied to Windows hosts, this module would modify the required registry values to configure the Windows time service to use our NTP servers, and make sure the service was started.

Modules can be constructed to perform a variety of tasks, not just package installation and file modification. You can also place entire files in specific directories and construct templates using the Ruby-based ERB language.

However, it is of paramount importance that any Puppet implementation have a testing and development sandbox. Each configuration element incorporates a significant number of moving parts and needs to be tested thoroughly before rolling out to production. You can accomplish this by maintaining a set of virtual servers that represent their production counterparts, and testing new modules and modifications only on the appropriate test node or group.

Puppet Enterprise also includes the Cloud Provisioner, which is an extension designed to work with Amazon's EC2 cloud service. It provides a way to create a new EC2 instance, install the Puppet agent, and sign the certificate from the command line on a Puppet Master server.

From the Live Management tab, you can run specific tasks on nodes such as installing a package or checking for package updates.

Reports, inventory, and Live ManagementThe Web UI also provides reporting and inventory features. Through a tabbed UI, you can view not only at-a-glance information on how Puppet itself is running on all nodes, but also detailed info on users, groups, installed packages, and running services on any given selection of nodes.

You can also perform certain tasks across nodes or groups of nodes from Live Management. Aside from enabling, disabling, and running the Puppet agent on demand, you can cause a node or group of nodes to install a package, start a service, or stop a service, or you can use apt or yum to update the server. There are also facilities to perform queries of the agent on a node or various rpcutil tasks, such as retrieving individual configuration items.

As far as the Puppet reporting goes, you get a tabular display of nodes and their run status. You might see a few nodes with failed updates, for example, and you can quickly dig into the logs to see what failed to apply. You can also get detailed reports on each agent run that provide in-depth info on what did or did not happen during that run. In addition, you can see how long it took to make the run, how many modifications were made, and so forth.

You can also search through inventory items across nodes. If you wanted to display a list of all nodes running kernel version 2.6.32, for instance, you would simply query for inventory item kernelversion = 2.6.32. By default Puppet collects a ton of inventory data, from BIOS version to system uptime to swap space to SELinux policy enforcement.

Powerful means complexPuppet is not for the faint of heart, nor should it be. Shepherding the configurations of a wide range of services and elements, managing package installation and removal, and collecting inventory data from dozens, hundreds, or thousands of servers is an inherently difficult task. Add to that the ability to heavily customize existing modules and to develop your own modules according to your own specifications, and you have a recipe that requires a great deal of complexity. Trying to obscure that complexity with overly simplistic, high-level configuration interfaces might make easy things easier, but it would make hard things impossible.

Puppet can be scaled out using multiple master servers and a variety of load-balancing methods. The fun part here is that you can use Puppet to manage your Puppet Master servers as well, allowing for synchronization across multiple masters. Plus, although you may have many master servers, you would still want a single reporting server to collect data from the masters.

Puppet Enterprise is not intended for the casual user or a small infrastructure. (In those cases, the small-scale version, Puppet Solo, might be a better fit.) Puppet Enterprise is intended for infrastructures with large numbers of similar servers, both physical and virtual. It can be used to manage switches, routers, and storage arrays, and it provides a reasonable means to handle them all. If you're looking for a quick-fix, clickable, GUI-based configuration management framework, this isn't it. If you're looking for a solid, scalable solution that provides plenty of access for deep custom development, you might find Puppet Enterprise to be the right choice.

This article, "Review: Puppet Enterprise 3.0 pulls more strings," was originally published at InfoWorld.com. Follow the latest developments in application development, cloud computing, and open source at InfoWorld.com. For the latest business technology news, follow InfoWorld.com on Twitter.

Read more about data center in InfoWorld's Data Center Channel.

Join the CIO Australia group on LinkedIn. The group is open to CIOs, IT Directors, COOs, CTOs and senior IT managers.

Join the newsletter!

Or

Sign up to gain exclusive access to email subscriptions, event invitations, competitions, giveaways, and much more.

Membership is free, and your security and privacy remain protected. View our privacy policy before signing up.

Error: Please check your email address.

Tags cloud computinginternetprivate cloudsoftwareData Centerhardware systemssystems managementsystem managementConfiguration / maintenancePuppet LabsServer Provisioning

More about Amazon Web ServicesDebianForge GroupLinuxOracleRedHatRed HatUbuntu

Show Comments
[]