Scheduled Outages or Maintenance Windows

Owned by Nick Day (Unlicensed)
Last updated: Dec 21, 2018 by Keith Sinclair (Deactivated)

If you have a maintenance window or a scheduled outage for a device then you will likely want to suspend alerting for that device during that period.
NMIS has supported this for a long time; NMIS 8.6.2 adds a number of new capabilities, in terms of scheduling such maintenance windows in advance or even recurringly.

Please note that the scope of a scheduled outage is the whole device, ie. no alerting whatsoever takes place for any aspect of the device for the duration of the outage.
Outages do not affect the polling of a device; NMIS does keep track of the device's status and any collected information even during an outage, it just won't produce alerts.

Managing Outage Windows using the GUI

  • In the NMIS GUI Select "Service Desk" -> "Alerts" -> "Outages"  

  • Select the device or devices involved.
  • Set the start and end time of the outage window.
  • Insert any reference number if required in the ticket number field



You can also delete outages using the GUI. You can, at this time, not update outages in the GUI.

How it works

  • NMIS checks for current outages whenever alerts or escalations are processed; If an outage is current, then alerting is suppressed.
  • NMIS does, however, not suspend polling during an outage.
  • While an outage is current and a node does not respond to ICMP or SNMP, then the KPI's for that poll are set to U (unknown, which prevents them from contributing to averages), this is so the overall reachability and availability results are not reduced for that node which has a planned outage.  The KPI's included in this exclusion are:
    • reachability
    • availability
    • intfTotal
    • responsetime
    • health
    • loss
  • Furthermore, the state of nodes with current outages does not contribute to the overall health metrics and KPIs, which helps with reporting statistics so your teams performance will look better!
  • In version 8.6.2 and newer, the per-node status view displays current and scheduled/future outages prominently.
  • In version 8.6.2 and newer, the events Planned Outage Open and Planned Outage Closed are raised for each polled device that enters or leaves an outage window.
  • In version 8.6.2 and newer NMIS tracks the state of outages per node and shows a translucent overlay on the node health graph for each outage.

Recurring Outages and Flexible Selectors (8.6.2 and newer)

In version 8.6.2 it is possible to schedule outages with much more flexibility, but that requires the use of the command line tool outage_admin.pl (see next section).

Recurring Outages

NMIS supports four frequency types: once, daily, weekly and monthly.
Each has its own start and end types, which follow the conventions for opReports' period and frequency format:

 Frequency
Format for Start and End
Explanation
Example
oncevariousAny of our Supported Time Formats should work,
but using the ISO8601 format  is the most robust choice		14:00 last monday
2015-03-30T16:31:53
weeklyWday HH:MM:SS
Wday HH:MM		Wday is one of  "Mon", "Tue" ... "Sun" (Case-insensitive).
Monday is considered first, Sunday last.
Start: Sun 14:00, End Wed 17:00 will cover sun, mon, tue, wed;
Start: Fri 17:00, End Mon 09:00 will cover fri, sat, sun, mon.		Fri 14:45
monthly

D HH:MM:SS
-D HH:MM:SS
D HH:MM
-D HH:MM

D is the day of the month, 1..31.  -D counts from the end of the month;
-1 is the last day of the month, -2 the second to last etc.		4 17:00
-1 00:00
dailyHH:MM:SS
HH:MM		24:00 means the end of the day, and makes sense only as period end.
00:00 means the beginning of the day. Leading zeros can be omitted.		14:15
23:45

 

Flexible Selectors

NMIS Versions before 8.6.2 support only selection of one or more nodes by name; in 8.6.2 you can use any node configuration property to determine which devices should be subject to your outage window.
Please note that the NMIS GUI does not expose any of these advanced selectors; you have to use outage_admin.pl to make use of them.

The selectors are given in the format of outage.node.<propertyname>, and one of three comparison operations are supported:

  1. Explicit Single Value
    e.g. outage.node.group=MyGroupName would select nodes that belong to group MyGroupName
  2. List of Explicit Values, as individual array elements
    e.g. outage.node.nodeModel.0=net-snmp outage.node.nodeModel.1=RedBack would select nodes whose model is either net-snmp or Redback
  3. Regular Expression
    e.g. outage.node.roleType=/^devel/ would select nodes whose configured roleType starts with "devel".

Managing Outages from the Command Line (8.6.2 and newer)

NMIS version 8.6.2 introduces the tool admin/outage_admin.pl, which lets you perform all outage-related operations. Simply start it without arguments and you'll be given an overview of the supported operations, like this:

/usr/local/nmis8/admin/outage_admin.pl 
Usage: outage_admin.pl act=[action to take] [extras...]
    outage_admin.pl act=list [filter=X...]
    outage_admin.pl act=create [outage.A=B... outage.X.Y=Z...]
    outage_admin.pl act=update id=<outid> [outage.A=B... outage.X.Y=Z...]
    outage_admin.pl act={delete|show} id=<outid>
    outage_admin.pl act=check [node=X] [time=T]
list: shows overview of defined outage schedules
show: displays the details for an outage
create: creates new outage schedule
 for detailed help, run ./admin/outage_admin.pl act=create
update: updates existing outage schedule
 only the given outage.A, outage.X.Y properties are changed.
check: reports which outages would apply at the 
 given time (or now) and  for one node (if given) or all nodes

To see the required/possible/expected arguments for outage creation, run it with act=create but no creation arguments, like this:

/usr/local/nmis8/admin/outage_admin.pl act=create
Supported Arguments for Outage Creation:
outage.description: free-form textual description.
outage.change_id: change management ticket identifier, used for event tagging
outage.frequency: one of 'once', 'daily', 'weekly' or 'monthly'
outage.start, outage.end: date and time of outage start and end,
 format depends on frequency
  daily: "HH:MM" or "HH:MM:SS". 24:00 is allowed for end.
  weekly: "MDAY HH:MM" or "MDAY HH:MM:SS", MDAY one of 'Mon', 'Tue' etc.
  monthly: "D HH:MM:SS", "-D HH:MM:SS", "D HH:MM", "-D HH:MM"
   D is the numeric day of the month, 1..31.  -D counts from the end of the month,
   -1 is the last day of the month, -2 the second to last etc.
  once: ISO8601 date time recommended,
   e.g. 2017-10-31T03:04:26+0000
outage.options: optional key=values to adjust NMIS' behaviour during an outage
outage.selector: any number of criteria for selecting devices for this outage
  selector keys: node.X or config.Y, node config or global config properties
  selector values: single string, /regex string/ or array or single strings.
  arrays must be given as separate indexed entries.
  all selectors must match for a node to be subject to the outage.
example: ./admin/outage_admin.pl act=create \
outage.description='certain nodes are busy each month start' \
outage.change_id='ticket #42' \
outage.frequency=monthly outage.start="1 12:00" outage.end="1 13:30" \
outage.selector.node.group.0="busybodies" \
outage.selector.node.group.1="alsobad"