Nagios notification WhatsApp Web

Flexible notifications for Nagios

Introduction


One of the core functions in an availability monitoring system is the ability to send out status alerts and notifications. Nagios only comes with a set of 2 default notifications through e-mails: notify-host-by-email other notify-service-by-email. These notifications provide a minimalistic, barebone structure for sending a simple e-mail alert to defined contacts when problems occur (example screenshot below).

Notification requirements


However, this minimal setup if short and the Nagios built-in format is not presentable to customers. Notification requirements in large companies are diverse and complex: Below are the critical ones I have received:

  • Notifications should support the use of the (cc :) field.
  • Notifications should include Nagios comments and author information, if given
  • Notifications should support different languages, based on the contact
  • Notifications should provide URL's back to Nagios, color-highlighting and the integration of logos

The reasons behind these requirements are as varied as todays IT organizations are:

  • Some notifications that go to customers or are escalated to management need to look "presentable and professional" - means pretty
  • The inclusion of Nagios GUI URL's linking back to the Nagios event speeds up the troubleshooting and follow-up by the IT support teams
  • Some notifications go out to outsourced IT service providers and should support their native language, the inclusion of additional information (such as contract number) and the exclusion of other details (such as device IP)
  • Some notifications are handled by endusers e-mail processing rules and must be plaintext or addressed through other address fields then (to :)

Thanks to Nagios flexibility and documentation, we are able to meet all requirements with a little extra work (click example screenshots below to enlarge).

 

Review of the initial notification setup


The initial Nagios e-mail notification definition is in. There, a minimum set of Nagios macros is combined into a message string and then piped into the standard UNIX / Linux mail client program mail or mailx. Because the mailx program expects to get the e-mail text in Unix format with lines separated by newline (\ n) characters, the shell command printf uses the formatting option % b to generate these newlines by interpreting the given (\ n) escape characters. Below is the original definition, with the original command_line definition being separated into multiple lines for better readability. Be careful with the backslash, a single missing space can ruin the whole command and prevent notifications from being send out.

First modifications


Before we look at tweaking the e-mail notification style, I make 3 basic changes to the overall setup.

  1. I move the notification definition from the original commands.cfg file into its own dedicated configuration file called notifications.cfg.
  2. I create an additional configuration file no-notification.cfg with identical definition names, but a command_line containing only a sleep 1 command (example definition below). This file is commented out in nagios.cfg and will be enabled a few times a year when the Nagios system is patched, rebooted or otherwise in maintenance. This simple measure avoids flooding our customers with false notifications when Nagios is brought up. After a longer downtime, it could take a while until services are re-checked and we learned to always do a sanity health check on the web GUI before we re-enable notifications.
  3. Lastly, I created 2 external Perl scripts responsible for sending out mail in place of the original printf / mailx definition

With these 2 newly developed scripts (nagios_send_host_mail.pl and nagios_send_service_mail.pl, one for host notifications and a separate for service notifications), it is possible to control through flags which format the email should be sent in, which language should be used and what information should be handed off for mailing out. Below I describe example setups that addressed my clients requirements.

Implementing carbon copy cc: notifications


One customer requested to receive Nagios notifications for some contacts to be defined as a carbon copy (cc :), instead of the standard e-mail recipient list in the (to :) field. The normal way of e-mail address hand-off is through the Nagios $ CONTACTEMAIL $ macro definition. Since this macro is already used for the (to :) field, We used the first of the six additionally available contact macros addressx. I defined address1 to hold the (cc :) e-mail address list.

Lets look at an example: We want to send alerts in plain text format to a mailing list [email protected] using English, and keep two others, [email protected] other [email protected] on (cc :).

Now we can add the $ CONTACTADDRESS1 $ macro into the notification definition located in notification.cfg to receive the (cc :) address list we placed in contacts.cfg.

Example:

Implementing HTML, links, comments and a second language


Here is a service notification example that takes advantage of the scripts ability to determine if comments have been entered in Nagios, and sending the notification in HTML format using Japanese language for the alert field descriptions. First, we set the definition format in the users contact settings to be HTML, use URL's to Nagios GUI for intranet access, and set the language to be Japanese.

Now we look at the corresponding notification definition in notification.cfg. The definition uses the format argument -f html, -u (include URL's) and the language argument -l jp. For more information about the arguments, see the scripts manual page for nagios_send_service_mail.pl.

Example:

Adding images by using the multipart S / MIME e-mail format


Here is a service notification example that sends the e-mail in multipart S / MIME with a logo image displayed inline within the HTML code. The example also adds in the company name using the -p argument. Currently, the logo image is defined in the script as it does not change for me. It can be changed by creating a uuencoded version of a different logo image, instructions are inside the script. In a future version, it could be implemented to be picked up from a filename using a script argument if necessary.

Now we look at the corresponding notification definition in notification.cfg. The definition uses the format argument -f multi, -u (include URL's) and ommiting language argument -l sets the language to English. For more information about the arguments, see the scripts manual page for nagios_send_service_mail.pl.

Example:

Adding the matching Nagiosgraph performance image, if available


Since version v1.4, the notification scripts are able to include the related performance graph image into the message. This greatly enhances the information provided inside the alert e-mail. However, this graph generation is dependent on the graph generation package used. It has been developed with Nagiosgraph v0.9.1, which I am using. The scripts work fine even with Nagiosgraph not being available, simply using the notification options 'text', 'html' and 'multi', leaving out the 'graph' option.

Example:

Downloads and Links


Nagios host and service notifications are handled through a dedicated script for each case, resulting in two scripts to download and install. The set below can provide embedded graph images from the popular nagiosgraph package (e.g. nagiosgraph 0.91).

nagios_send_service_mail.pl

nagios_send_service_mail.pl Latest version 1.8.1, 48814 Bytes, for further documentation see also the manual page

nagios_send_service_mail-v180.pl Obsolete version 1.8.0, 46712 bytes

nagios_send_service_mail-v173.pl Obsolete version 1.7.3, 45162 bytes

nagios_send_service_mail-v14.pl Obsolete version 1.4.0, 41368 bytes

nagios_send_service_mail-v13.pl Obsolete version 1.3.0, 31664 bytes

nagios_send_host_mail.pl

nagios_send_host_mail.pl Latest version 1.8.1, 47328 bytes, for further documentation see also the manual page

nagios_send_host_mail-v180.pl Obsolete version 1.8.0, 45490 bytes

nagios_send_host_mail-v173.pl Obsolete version 1.7.3, 43891 bytes

nagios_send_host_mail-v14.pl Obsolete version 1.4.0, 38330 bytes

nagios_send_host_mail-v13.pl Obsolete version 1.3.0, 28477 bytes

For an alternate download location, see also the corresponding github repository.

Notification Script Contributions

SurnameVer.Description
nagios_send_host_mail_rb.pl1.4The original host notification script v1.4, with the addition of the extra option '-g' to use the Nagios contact groups.
nagios_send_service_mail_rb.pl1.4The original service notification script v1.4, with the extra option '-g' to use the Nagios contact groups.
nagios_send_service_mail_rb.pl1.6The service notification script v1.6 (March 2012) implements various enhancements, such as improved multi-language support, the addition of German and French, enhanced debugging, etc.
Many Thanks to Robert Becht for the contribution.

Below are two example notification definition files:

Similar Solutions, Links and Credits:

If you made cool changes to the scripts, fixed bugs or expanded the functions, feel free to send them in, together with a screenshot, for sharing.

Known bugs


NOTE: The notification scripts fail on Nagios v4.0 due to a Nagios bug (BugID 0000498). The Nagios environment variables are not exported, and as a result the data handover to the scripts does not work. No notifications go out since no destination e-mail address comes from Nagios. This problem is fixed in Nagios 4.0.3

Support for PNP4Nagios


After the initial release of these scripts they became very popular. However many of you did not use the Nagiosgraph package, but the more popular choice PNP4Nagios. The question about adapting these scripts for PNP4Nagios was raised many times, and a lively discussion started. I relented to public pressure :-) I installed PNP4Nagios, built a parallel datafeed into both graphing systems and started to spend quite a few nights over code and testing, receiving invaluable help from Robert Becht. The result is a pair of notification scripts dedicated to PNP4Nagios: pnp4nagios_send host_mail.pl other pnp4nagios_send_service_mail.pl.

nagios_send_service_mail.pl

pnp4n_send_service_mail.pl Latest version 1.8.1, 45474 Bytes, for further documentation see also the manual page

pnp4n_send_service_mail-v180.pl Obsolete version 1.8.0, 43369 bytes

pnp4n_send_service_mail-v173.pl Obsolete version 1.7.3, 41850 bytes

pnp4n_send_service_mail-v15.pl Obsolete version 1.5.0, 41533 bytes

pnp4n_send_service_mail-v14.pl Obsolete version 1.4.0, 40968 bytes

pnp4n_send_host_mail.pl

pnp4n_send_host_mail.pl Latest version 1.8.1, 44172 bytes

pnp4n_send_host_mail-v180.pl Obsolete version 1.8.0, 42333 bytes

pnp4n_send_host_mail-v173.pl Obsolete version 1.7.3, 40778 bytes

pnp4n_send_host_mail-v15.pl Obsolete version 1.5.0, 38019 bytes

pnp4n_send_host_mail-v14.pl Obsolete version 1.4.0, 37926 bytes

Topics:

More information: