Sub heading - How to use a Nagios plugin for service monitoring

Related Pages

Managing Servers and Services with NMIS8

Purpose

Nagios may have a plugin available that we'd like to leverage with NMIS. NMIS is capable of integrating Nagios plugins. This article will provide an example of how to do this to check the expiration date of an SSL certificate.

Steps

Determine which Nagios plugin to utilize
Find and Install the package containing the plugin
Test the plugin from the command line
Create NMIS services that utilize the Nagios plugin
Assign a Service to an NMIS node
Check the GUI

Determine Which Nagios Plugin to Utilize

Here are two pages that have have Nagios plugins.

https://www.monitoring-plugins.org/

http://nagios-plugins.org/

For this example we'll use the check_http plugin.

Find and Install The Package that Contains the Plugin

Since the NMIS VM runs on CentOS we'll use Yum as an example. For Debian based systems please see Appendix B for command examples.

[root@opmantek ~]# yum search check_http
Loaded plugins: fastestmirror, presto
Loading mirror speeds from cached hostfile
epel/metalink                                                                                                            | 7.0 kB     00:00     
 * base: ftp.kaist.ac.kr
 * epel: mirror.premi.st
 * extras: ftp.kaist.ac.kr
 * updates: ftp.kaist.ac.kr
base                                                                                                                     | 3.7 kB     00:00     
epel                                                                                                                     | 4.3 kB     00:00     
epel/primary_db                                                                                                          | 5.9 MB     00:00     
extras                                                                                                                   | 3.4 kB     00:00     
gf                                                                                                                       | 2.9 kB     00:00     
mongodb-org-3.0                                                                                                          | 2.5 kB     00:00     
rsyslog_v7                                                                                                               | 2.5 kB     00:00     
updates                                                                                                                  | 3.4 kB     00:00     
=========================================================== N/S Matched: check_http ============================================================
nagios-plugins-http.x86_64 : Nagios Plugin - check_http
  Name and summary matches only, use "search all" for everything.

In the example above we search the Yum repositorys for check_http and discover it is in the nagios-plugins-http.x86_64 package. Next install it.

[root@opmantek ~]# yum install nagios-plugins-http.x86_64
Loaded plugins: fastestmirror, presto
Setting up Install Process
Loading mirror speeds from cached hostfile
 * base: ftp.kaist.ac.kr
 * epel: mirror.premi.st
 * extras: ftp.kaist.ac.kr
 * updates: ftp.kaist.ac.kr
Resolving Dependencies
--> Running transaction check
---> Package nagios-plugins-http.x86_64 0:2.1.4-3.el6 will be installed
--> Finished Dependency Resolution
Dependencies Resolved
================================================================================================================================================
 Package                                    Arch                          Version                             Repository                   Size
================================================================================================================================================
Installing:
 nagios-plugins-http                        x86_64                        2.1.4-3.el6                         epel                         56 k
Transaction Summary
================================================================================================================================================
Install       1 Package(s)
Total size: 56 k
Installed size: 137 k
Is this ok [y/N]: y
Downloading Packages:
Running rpm_check_debug
Running Transaction Test
Transaction Test Succeeded
Running Transaction
  Installing : nagios-plugins-http-2.1.4-3.el6.x86_64                                                                                       1/1 
  Verifying  : nagios-plugins-http-2.1.4-3.el6.x86_64                                                                                       1/1 
Installed:
  nagios-plugins-http.x86_64 0:2.1.4-3.el6                                                                                                      
Complete!

Very well, but where is the plugin? We will need the full path to the plugin when we provision the NMIS service.

[root@opmantek ~]# rpm -ql  nagios-plugins-http.x86_64
/usr/lib64/nagios/plugins/check_http

The installer has placed it in /usr/lib64/nagios/plugins. Let's take a look at what else it has installed.

[root@opmantek /]# cd /usr/lib64/nagios/plugins/
[root@opmantek plugins]# ls
check_http  negate  urlize  utils.sh
[root@opmantek plugins]# ls -l
total 236
-rwxr-xr-x 1 root root 140552 2017-01-18 10:28 check_http
-rwxr-xr-x 1 root root  46464 2017-01-18 10:28 negate
-rwxr-xr-x 1 root root  44352 2017-01-18 10:28 urlize
-rwxr-xr-x 1 root root   2791 2017-01-18 10:28 utils.sh

There were three other plugins in the package. These are utility plugins that can invert a response or check if the response falls within a range. To check how the check_http plugin works run it with the --help flag. An example of this can be found in Appendix A of this article. The help output states check_http should be run with the -C flag in order to check SSL certificate status.

-C <warn_age>[,<crit_age>]

With a command such as:

[root@opmantek ~]# /usr/lib64/nagios/plugins/check_http -I demo.opmantek.com -C 30, 15

If the certificate expiration date is greater than 30 days away it will return OK, warning will be returned if the expiration date is less than 30 days away, critical will be returned if the expiration date is less than 15 days away.

Test the Plugin From the Command Line

This is important because when we provision the NMIS service latter we need to have the correct syntax for the plugin. In this example we use the plugin to check if the certificate is going to expire within the next 30 days.

[root@opmantek ~]# /usr/lib64/nagios/plugins/check_http -I demo.opmantek.com -C 30, 15
OK - Certificate '*.opmantek.com' will expire on 2017-11-16 15:16 +0900/KST.

Observe that the -C flag will check the certificates expiration date and provide a warning if it's less that 30 days away.

Create and Assign a Service to An NMIS Node

Create the Service

Via the top menu bar navigate to System -> System Configuration -> Services. A new window 'Services' will render. In the top right hand corner of this new window click 'add'. A window such as the one below will appear.

Notice in the example above we have populated the Name, Service Type, Program Path, Program Args, and Collect Program Ouput fields.

Assign Service To NMIS Node

Via the top menu bar navigate to System -> System Configuration > NMIS Nodes. A new 'NMIS Nodes' window will render. For this example we will assign the checkDemoCert to the node localhost. Find the node localhost and click 'edit' on the right hand side. A new window will render. Find the Advanced Options Section. Creatively using the shift and mouse left clicker, select the service checkdemocert, careful not to de-select any other provisioned servcies.

At the bottom of the window click the 'Edit and Update Node' button. After some processing time a new window will render with debug detail of the update process. Feel free to close this window.

Check the GUI

On the left side of the page find the 'Quick Search' box. In the middle of this box enter 'localhost' in the 'Filter Device list by input string' field. localhost should appear in the box below, click on it. A new local host window will render. using the top menu bar of this new window click on 'services'. All the currently provisioned services will render. On the left hand side click on the new service 'checkdemocert'. A new window will render like the one below.

Notice it displays that the Certificate is OK because it will not expire within the next 30 days. In the 'Last Status Text' field it displays the current expiration date.

Appendix A - check_http Documentation

[root@opmantek plugins]# ./check_http --help
check_http v2.1.4 (nagios-plugins 2.1.4)
Copyright (c) 1999 Ethan Galstad <nagios@nagios.org>
Copyright (c) 1999-2014 Nagios Plugin Development Team
        <devel@nagios-plugins.org>

This plugin tests the HTTP service on the specified host. It can test
normal (http) and secure (https) servers, follow redirects, search for
strings and regular expressions, check connection times, and report on
certificate expiration times.


Usage:
 check_http -H <vhost> | -I <IP-address> [-u <uri>] [-p <port>]
       [-J <client certificate file>] [-K <private key>]
       [-w <warn time>] [-c <critical time>] [-t <timeout>] [-L] [-E] [-a auth]
       [-b proxy_auth] [-f <ok|warning|critcal|follow|sticky|stickyport>]
       [-e <expect>] [-d string] [-s string] [-l] [-r <regex> | -R <case-insensitive regex>]
       [-P string] [-m <min_pg_size>:<max_pg_size>] [-4|-6] [-N] [-M <age>]
       [-A string] [-k string] [-S <version>] [--sni] [-C <warn_age>[,<crit_age>]]
       [-T <content-type>] [-j method]
NOTE: One or both of -H and -I must be specified

Options:
 -h, --help
    Print detailed help screen
 -V, --version
    Print version information
 --extra-opts=[section][@file]
    Read options from an ini file. See
    https://www.nagios-plugins.org/doc/extra-opts.html
    for usage and examples.
 -H, --hostname=ADDRESS
    Host name argument for servers using host headers (virtual host)
    Append a port to include it in the header (eg: example.com:5000)
 -I, --IP-address=ADDRESS
    IP address or name (use numeric address if possible to bypass DNS lookup).
 -p, --port=INTEGER
    Port number (default: 80)
 -4, --use-ipv4
    Use IPv4 connection
 -6, --use-ipv6
    Use IPv6 connection
 -S, --ssl=VERSION[+]
    Connect via SSL. Port defaults to 443. VERSION is optional, and prevents
    auto-negotiation (2 = SSLv2, 3 = SSLv3, 1 = TLSv1, 1.1 = TLSv1.1,
    1.2 = TLSv1.2). With a '+' suffix, newer versions are also accepted.
 --sni
    Enable SSL/TLS hostname extension support (SNI)
 -C, --certificate=INTEGER[,INTEGER]
    Minimum number of days a certificate has to be valid. Port defaults to 443
    (when this option is used the URL is not checked.)
 -J, --client-cert=FILE
   Name of file that contains the client certificate (PEM format)
   to be used in establishing the SSL session
 -K, --private-key=FILE
   Name of file containing the private key (PEM format)
   matching the client certificate
 -e, --expect=STRING
    Comma-delimited list of strings, at least one of them is expected in
    the first (status) line of the server response (default: HTTP/1.)
    If specified skips all other status line logic (ex: 3xx, 4xx, 5xx processing)
 -d, --header-string=STRING
    String to expect in the response headers
 -s, --string=STRING
    String to expect in the content
 -u, --uri=PATH
    URI to GET or POST (default: /)
 --url=PATH
    (deprecated) URL to GET or POST (default: /)
 -P, --post=STRING
    URL encoded http POST data
 -j, --method=STRING  (for example: HEAD, OPTIONS, TRACE, PUT, DELETE, CONNECT)
    Set HTTP method.
 -N, --no-body
    Don't wait for document body: stop reading after headers.
    (Note that this still does an HTTP GET or POST, not a HEAD.)
 -M, --max-age=SECONDS
    Warn if document is more than SECONDS old. the number can also be of
    the form "10m" for minutes, "10h" for hours, or "10d" for days.
 -T, --content-type=STRING
    specify Content-Type header media type when POSTing

 -l, --linespan
    Allow regex to span newlines (must precede -r or -R)
 -r, --regex, --ereg=STRING
    Search page for regex STRING
 -R, --eregi=STRING
    Search page for case-insensitive regex STRING
 --invert-regex
    Return CRITICAL if found, OK if not

 -a, --authorization=AUTH_PAIR
    Username:password on sites with basic authentication
 -b, --proxy-authorization=AUTH_PAIR
    Username:password on proxy-servers with basic authentication
 -A, --useragent=STRING
    String to be sent in http header as "User Agent"
 -k, --header=STRING
    Any other tags to be sent in http header. Use multiple times for additional headers
 -E, --extended-perfdata
    Print additional performance data
 -L, --link
    Wrap output in HTML link (obsoleted by urlize)
 -f, --onredirect=<ok|warning|critical|follow|sticky|stickyport>
    How to handle redirected pages. sticky is like follow but stick to the
    specified IP address. stickyport also ensures port stays the same.
 -m, --pagesize=INTEGER<:INTEGER>
    Minimum page size required (bytes) : Maximum page size required (bytes)
 -w, --warning=DOUBLE
    Response time to result in warning status (seconds)
 -c, --critical=DOUBLE
    Response time to result in critical status (seconds)
 -t, --timeout=INTEGER:<timeout state>
    Seconds before connection times out (default: 10)
    Optional ":<timeout state>" can be a state integer (0,1,2,3) or a state STRING
 -v, --verbose
    Show details for command-line debugging (Nagios may truncate output)

Notes:
 This plugin will attempt to open an HTTP connection with the host.
 Successful connects return STATE_OK, refusals and timeouts return STATE_CRITICAL
 other errors return STATE_UNKNOWN.  Successful connects, but incorrect reponse
 messages from the host result in STATE_WARNING return values.  If you are
 checking a virtual server that uses 'host headers' you must supply the FQDN
 (fully qualified domain name) as the [host_name] argument.

 This plugin can also check whether an SSL enabled web server is able to
 serve content (optionally within a specified time) or whether the X509
 certificate is still valid for the specified number of days.

 Please note that this plugin does not check if the presented server
 certificate matches the hostname of the server, or if the certificate
 has a valid chain of trust to one of the locally installed CAs.

Examples:
 CHECK CONTENT: check_http -w 5 -c 10 --ssl -H www.verisign.com

 When the 'www.verisign.com' server returns its content within 5 seconds,
 a STATE_OK will be returned. When the server returns its content but exceeds
 the 5-second threshold, a STATE_WARNING will be returned. When an error occurs,
 a STATE_CRITICAL will be returned.

 CHECK CERTIFICATE: check_http -H www.verisign.com -C 14

 When the certificate of 'www.verisign.com' is valid for more than 14 days,
 a STATE_OK is returned. When the certificate is still valid, but for less than
 14 days, a STATE_WARNING is returned. A STATE_CRITICAL will be returned when
 the certificate is expired.


 CHECK CERTIFICATE: check_http -H www.verisign.com -C 30,14

 When the certificate of 'www.verisign.com' is valid for more than 30 days,
 a STATE_OK is returned. When the certificate is still valid, but for less than
 30 days, but more than 14 days, a STATE_WARNING is returned.
 A STATE_CRITICAL will be returned when certificate expires in less than 14 days
 CHECK SSL WEBSERVER CONTENT VIA PROXY USING HTTP 1.1 CONNECT:

 check_http -I 192.168.100.35 -p 80 -u https://www.verisign.com/ -S -j CONNECT -H www.verisign.com
 all these options are needed: -I <proxy> -p <proxy-port> -u <check-url> -S(sl) -j CONNECT -H <webserver>
 a STATE_OK will be returned. When the server returns its content but exceeds
 the 5-second threshold, a STATE_WARNING will be returned. When an error occurs,
 a STATE_CRITICAL will be returned.

Send email to help@nagios-plugins.org if you have questions regarding use
of this software. To submit patches or suggest improvements, send email to
devel@nagios-plugins.org

Appendix B - Debian based systems - command line examples.

Finding the package that contains the subject plugin.

root@DebTest:~# apt-cache search check_http
monitoring-plugins-basic - Plugins for nagios compatible monitoring systems (basic)
nagios-plugins-contrib - Plugins for nagios compatible monitoring systems

Installing the package.

root@DebTest:~# apt-get install monitoring-plugins-basic
Reading package lists... Done
Building dependency tree       
Reading state information... Done
The following packages were automatically installed and are no longer required:
  liboath0 libopenconnect3 libproxy1 libstoken1 libtomcrypt0 libtommath0 vpnc-scripts
Use 'apt-get autoremove' to remove them.
The following extra packages will be installed:
  monitoring-plugins-common
Suggested packages:
  icinga icinga nagios3
The following NEW packages will be installed:
  monitoring-plugins-basic monitoring-plugins-common
0 upgraded, 2 newly installed, 0 to remove and 81 not upgraded.
Need to get 668 kB of archives.
After this operation, 2,233 kB of additional disk space will be used.
Do you want to continue? [Y/n] y
Get:1 http://ftp.kr.debian.org/debian/ jessie/main monitoring-plugins-common amd64 2.1.1-1 [223 kB]
1% [1 monitoring-plugins-common 8,451 B/223 kB 4%]   
-- snip

Discovering the location of check_http.

root@DebTest:~# dpkg -L monitoring-plugins-basic | grep check_http
/usr/lib/nagios/plugins/check_http