Tag Archives: nagios

Nagios Config Howto Followup

One of the most widely read stories I’ve ever posted on this blog is my Nagios Configuration HowTo, where I explained how I set up my Nagios config at a former employer. I still think that it’s a good layout to use if you’re manually building Nagios configs. In my current position, we have a small manual setup and a humongous automated monitoring setup. We’re moving toward a completely automated monitoring config using Puppet and Nagios, but until everything is puppetized, some of it needs to be hand-crafted, bespoke monitoring.

For people who don’t have a single ‘source of truth’ in their infrastructure that they can draw monitoring config from, hand-crafting is still the way to go, and if you’re going to do it, you might as well not drive yourself insane. For that, you need to take advantage of the layers of abstraction in Nagios and the built-in object inheritance that it offers.

Every once in a while, new content gets posted that refers back to my Config HowTo, and I get a bump in visits, which is cool. Occasionally, I’ll get someone who is interested and asks questions, which is what happened in this thread on Reddit. /u/sfrazer pointed to my config as something that he references when making Nagios configs (Thanks!), and the original submitter replied:

I’ve read that write up a couple of times. My configuration of Nagios doesn’t have an objects, this is what it looks like

(click to embiggen)

And to understand what you are saying, just by putting them in the file structure you have in your HowTo that will create an inheritance?

I wanted to help him understand how Nagios inheritance works, so I wrote a relatively long response, and I thought that it might also help other people who still need to do this kind of thing:


No, the directories are just to help remember what is what, and so you don’t have a single directory with hundreds of files.

What creates the inheritance is this:

You start out with a host template:

[email protected]:/usr/local/nagios/etc/objects$ cat generic-host.cfg
define host {
    name generic-host
    notifications_enabled   1
    event_handler_enabled   1
    flap_detection_enabled  1
    failure_prediction_enabled  1
    process_perf_data   1
    retain_status_information   1
    retain_nonstatus_information    1
    max_check_attempts 3
    notification_period 24x7
    contact_groups systems
    check_command check-host-alive
    register 0

So, what you can see there is that I have a host named “generic-host” with a bunch of settings, and “register 0”. The reason I have this is that I don’t want to have to set all of those settings for every other host I make. That’s WAY too much redundancy. Those settings will almost never change (and if we do have a specific host that needs to have the setting changed, we can do it on that host).

Once we have generic host, lets make a ‘generic-linux’ host that we can have the linux machines use:

[email protected]:/usr/local/nagios/etc/objects/linux$ cat generic-linux.cfg 
define host { 
    name     linux-server
    use generic-host
    check_period    24x7
    check_interval  5
    retry_interval  1
    max_check_attempts  5
    check_command   check-host-alive
    notification_interval 1440
    contact_groups  systems
    hostgroups linux-servers
    register 0

define hostgroup {
    hostgroup_name linux-servers
    alias Linux Servers

Alright, so you see we have two things there. A host, named ‘linux-server’, and you can see that it inherits from ‘generic-host’. I then set some of the settings specific to the monitoring host that I’m using (for instance, you probably don’t want notification_interval 1440, because that’s WAY too long for most people – a whole day would go between Nagios notifications!). The point is that I set a bunch of default host settings in ‘generic-host’, then did more specific things in ‘linux-server’ which inherited the settings from ‘generic-host’. And we made it ‘register 0’, which means it’s not a “real” host, it’s a template. Also, and this is important, you’ll see that we set ‘hostgroups linux-servers’. This means that any host we make that inherits from ‘linux-server’ will automatically be added to the ‘linux-servers’ hostgroup.

Right below that, we create the linux-servers hostgroup. We aren’t listing any machines. We’re creating an empty hostgroup, because remember, everything that inherits from linux-servers will automatically become a member of this group.

Alright, you’ll notice that we don’t have any “real” hosts yet. We’re not going to yet, either. Lets do some services first.

[email protected]:/usr/local/nagios/etc/objects$ cat check-ssh.cfg
define command{
   command_name   check_ssh
   command_line   $USER1$/check_ssh $ARG1$ $HOSTADDRESS$

This is a short file which creates a command called “check_ssh”. This isn’t specific to Linux or anything else. It could be used by anything that needed to verify that SSH was running. Now, lets build a service that uses it:

[email protected]:/usr/local/nagios/etc/objects/services$ cat generic-service.cfg 
define service{
        name                            generic-service  
        active_checks_enabled           1     
        passive_checks_enabled          1      
        parallelize_check               1       
        obsess_over_service             1        
        check_freshness                 0         
        notifications_enabled           1          
        event_handler_enabled           1           
        flap_detection_enabled          1            
        failure_prediction_enabled      1             
        process_perf_data               1
        retain_status_information       1 
        retain_nonstatus_information    1  
        is_volatile                     0   
        check_period                    24x7 
        max_check_attempts              3     
        normal_check_interval           10     
        retry_check_interval            2       
        contact_groups                  systems
      notification_options    w,u,c,r        
        notification_interval           1440
        notification_period             24x7       
         register                        0            

This is just a generic service template with sane settings for my environment. Again, you’ll want to use something good for yours. Now, something that will inherit from generic-service:

[email protected]:/usr/local/nagios/etc/objects/linux$ cat linux-ssh.cfg
define service { 
    use generic-service
    service_description Linux SSH Enabled
    hostgroup_name linux-servers
    check_command check_ssh 

Now we have a service “Linux SSH Enabled”. This uses check_ssh, and (importantly), ‘hostgroup_name linux-servers’ means “Every machine that is a member of the hostgroup ‘linux-servers’ automatically gets this service check”.

Lets do the same thing with ping:

define command{
        command_name    check_ping
        command_line    $USER1$/check_ping -H $HOSTADDRESS$ -w $ARG1$ -c $ARG2$ -p 5

 define service { 
    use generic-service
    service_description Linux Ping
    hostgroup_name linux-servers
    check_command check_ping!3000.0,80%!5000.0,100

Sweet. (If you’re wondering about the exclamation marks on the check_ping line in the Linux Ping service, we’re sending those arguments to the command, which you can see set the warning and critical thresholds).

Now, lets add our first host:

[email protected]:/usr/local/nagios/etc/objects/linux$ cat mylinuxserver.mycompany.com.cfg 
define host{
       use linux-server
       host_name myLinuxServer.mycompany.com
       address my.ip.address.here

That’s it! I set the host name, I set the IP address, and I say “use linux-server” so that it automatically gets all of the “linux-server” settings, including belonging to the linux host group, which makes sure that it automatically gets assigned all of the Linux service checks. Ta-Da!

Hopefully this can help people see the value in arranging configs like this. If you have any questions, please let me know via comments. I’ll be happy to explain! Thanks!



Nagios-Plugins Brouhaha

I’m really not a fan of political infighting. It’s bad for an organization, and it’s bad for the people who rely on the organization. But it happens, because we’re people, and people have ideas and egos and goals that are mutually exclusive of each other.

Such as it is with Nagios at the moment. Although there’s been some strife for the IP and trademarks surrounding Nagios for a while, the most recent thing is that the plugins site was…reassigned, I suppose you would say.

For a brief timeline, Nagios began as NetSaint, back in 1999. It was renamed Nagios in 2001 according to the WayBack Machine, apparently because of potential trademark issues. The plugins were apparently spun off from the main NetSaint project around this time as well, although the domain creation date is 2008-05-23 for nagios-plugins.org and 2007-01-15 for nagiosplugins.org.

So what’s going on now? Well, according to a news entry on the Nagios.org site, the Nagios Plugin Team had some changes:

The Nagios Plugin team is undergoing some changes, including the introduction of a new maintainer. The www.nagios-plugins.org website will remain the official location of the Nagios Plugins, and development of the plugins will continue on github at https://github.com/nagios-plugins.

Changes are being made to the team as the result of unethical behavior of the previous maintainer Holger Weiss. Weiss had repeatedly ignored our requests to make minor changes to the plugins website to reflect their relation to Nagios, rather than unrelated projects and companies. After failing to acknowledge our reasonable requests, we updated the website to reflect the changes we had requested. Rather than contacting us regarding the change, Weiss decided to embark on a vitriolic path of attacking Nagios and spreading mistruths about what had happened.

We believe that this type of unethical behavior is not beneficial for the Nagios community nor is it in keeping with the high standards people have come to rely on from Nagios. Thus, we have decided to find a new maintainer for the plugins. A new maintainer has already stepped forward and will be announced shortly.

We would like to thank all current and past plugin developers for their contributions and welcome anyone new who is interested in contributing to the project moving forward.

So that’s what Nagios has to say.

The reason that they specify the official location for the plugins is because the original team is continuing development on the (their?) project, at https://www.monitoring-plugins.org. According to a news post there:

In the past, the domain nagios-plugins.org pointed to a server independently maintained by us, the Nagios Plugins Development Team. Today, the DNS records were modified to point to web space controlled by Nagios Enterprises instead. This change was done without prior notice.

This means the project can no longer use the name “Nagios Plugins”. We, the Nagios Plugins Development Team, therefore renamed the Nagios Plugins to Monitoring Plugins.

We’re not too happy having to make this move. Renaming the project will lead to some confusion, and to quite a bit of work for others and for ourselves. We would’ve preferred to save everyone this trouble.

However, we do like how the new name indicates that our plugins are also used with various other monitoring applications these days. While the Nagios folks created the original implementation of the core plugins bundle, an independent team has taken over development more than a decade ago, and the product is intended to be useful for all users, including, but not limited to, the customers of Nagios Enterprises.

It’ll probably take us a few days to sort out various issues caused by the new project name, but we’re confident that we can resume our development work towards the next stable releases very soon.

We’d like to take the chance to thank you, our community, for your countless contributions, which made the plugins what they are today. You guys are awesome. We’re looking forward to the next chapter of Monitoring Plugins development, and we hope you are, too!

Throwing gasoline onto the fire is Michael Friedrich, Lead Developer of Icinga, a Nagios Fork, who submitted a RedHat bug claiming:

The nagios-plugins.org website has been compromised, and the project team therefore moved to https://www.monitoring-plugins.org including the tarball releases. They also renamed their project from ‘nagios-plugins’ to ‘monitoring-plugins’ and it’s most likely that the tarball release names will be changed to the new name in future releases.


Additional info:

While I wouldn’t suggest to rename the package unless there’s an immediate requirement, the source and URL location should be updated in order to use official releases provided by the Monitoring Plugins Development Team. Further, users should use official online references and stay safe.

then later in the thread

Actually the old Nagios Plugins Development Team was required to rename their project due to the fact that Nagios Enterprises hijacked the DNS and website and kicked them out.

Whilst the original memebers are now known as ‘Monitoring Plugins Development Team’, the newly formed ‘Nagios Core Plugin Development Team’ is actually providing a fork of their work under the old name.

For any clarifications required, you should follow the discussion here: https://www.monitoring-plugins.org/archive/devel/2014-January/009417.html

Imho the official former nagios plugins are now provided by the same developers under a new URL and that should be reflected for any future updates.

Though, I’m leaving that to you who to trust here. I’m just a community member appreciating the work done by the original Nagios Plugins Development team, accepting their origin and the fact that there’s no censorship of Icinga, Shinken or Naemon (as Nagios forks) needed.

Clearly, the nagios-plugins.org site was not compromised – neither Nagios nor the Monitoring Plugins team is claiming that. I’ll kindly assume that Mr Freidrich was mistaken when he posted the original bug. Hanlon’s Razor and all.

So what’s my opinion? Glad you asked.

The Nagios news post states that there had been continued requests to change small content. When that didn’t happen, they pulled the rug out from under half a dozen community contributors who have collectively done a great deal of good for the project. That’s not the way you show appreciation where I’m from, but hey, I don’t know the particulars. I only know what I see on the web, just like you.

What does this all mean for us? Well, if you run anything that uses Nagios plugins, it means you’ve got a choice – go with the official package, or go with the community-maintained version. Which will be better? Which will the distros use? Probably the official plugins, though I expect the more rapidly-moving distros to offer a package from the monitoring-plugin team as soon as there’s any noticeable difference.

But on the bigger picture scale, Nagios’s previously solid position as the principle Open Source monitoring solution isn’t as unassailable as it seems they think it is. Cutting off a volunteer team that produces a big part of your product isn’t really a good way to advertise stability and unification. There are a lot of options for monitoring today, and a lot more of them are way more viable than was the case 4 or 5 years ago. Instead of this political crap that does nothing to advance the project, I think Nagios should focus on improving the core product. But what do I know? I’m just some blogger on the internet.

Strangely, as I write this, the Nagios Exchange is down. I don’t know what that means.

Review: Nagios Core Administration Cookbook

I like Nagios. I mean, I like monitoring in general, but I like Nagios because it doesn’t necessarily tell me what I have to monitor, and it especially doesn’t tell me how I have to monitor it. It’s very extensible, and although it certainly has flaws, I think it’s a good, solid, reliable system that has cost me a great deal of sleep in my life as a system administrator.
Nagios Core Administration Cookbook
When Packt Publishing asked me to review Tom Ryder’s book, Nagios Core Administration Cookbook, I was happy to do it. When I first found Nagios, I learned by reading the example configuration files, and I sadly followed their lead for way too long. I eventually learned that there are better ways, but I like resources that help people to learn faster than I did.

Tom’s book follows the standard template (hah, template! Get it? that’s a little Nagios humor for you) for books introducing Nagios by talking about the differences in the basic object definitions, with extra weight going into the hosts, services, and contacts, arguably the three most important object types. From that point, he moves swiftly into the normal “Cookbook” method of short, easily digestible (hah! Digestible! There’s a cookbo…ok, I’ll stop) recipes to get things done.

I really like that Tom made the decision early to create separate configuration files for individual hosts. I can’t overstate how bad of an idea following the “hosts.cfg, services.cfg, contacts.cfg” examples are.

One of the things that I suppose was inevitable from the commercialization of Nagios is that there’s a necessary distinction between Nagios Core (the free product) and Nagios XI (the pay-to-play supported product). In reading this book, I noticed that Tom continually writes “Nagios Core” whenever referring to the free product. I understand that it is necessary to distinguish between the two, but I wish this could have been accomplished with a disclaimer up front, to the effect of, “Whenever I say Nagios, I mean Nagios Core unless I explicitly state otherwise”. It really only bothered me for the first chapter, then I started to glaze over it, but for the first chapter, it was kind of grating. It doesn’t harm the quality of the text, though.

When you read a review like this, you probably care about two things. One, did the reviewer recommend it. And two, should you buy it. So here’s my take.

This is a book targeting people who need to get things done with Nagios. This is not Wolfgang Barth’s Nagios: 2nd Edition (which will teach you everything about everything Nagios-related). It’s not trying to be. It’s trying to give you small bits of information as you need them. In that, it does a good job.

If you don’t have time to sit down and learn Nagios, but you want to get started with its configuration, then yes, you should pick this book up. You can teach yourself everything in it on your own, but it’s way more economical on your time to see how other people (like Tom) have done what you’re trying to do. His examples are good, short, and the comments are helpful. Also, I feel that he makes good choices in his recommendations on managing the configuration (which I personally think is much harder than actually writing the configuration). If you’ve had issues writing, organizing, or storing Nagios config, then you want to read all of the entries in Chapter 9.

Overall, the book has a lot of valuable recipes, and if you didn’t know Nagios inside and out, but walked into a job where there was a Nagios infrastructure that you had to deal with, then you could do a lot worse than buying this book. If you think it sounds like you could use this book, then I recommend you get it, because it is what it’s advertised to be: A Nagios cookbook.

Nagios Core Administration Cookbook is 366 pages long and includes over 80 recipes, full color screenshots, and well-commented example config and code. You can get the ebook from PacktPub for $25.49, and it comes in epub, mobi, and pdf formats. $49.99 will get you the physical book and a digital copy as well, if that’s your thing, too, but honestly, I’d save the money and get the digital copy.

Also, you can check out Tom Ryder’s blog at http://www.blog.sanctum.geek.nz