Creating custom Policies and Actions
Policies and Actions Concept
opEvents generates a variety of events and depending on the size and health of your network there may be thousands of events a day. These events can vary in importance from warnings of high CPU usage to critical failures of hardware. It is not necessary to send email notifications to every engineer on staff in regards to minor events. If every event is notifying everyone on staff then soon there will be less attention to these events making it easier to overlook the critical ones. opEvents gives you the ability to create policies, notify, and generate automated actions based on rules that make sense for your environment.
NOTE - This guide requires you to be familiar with Perl and Linux Operating Systems. Helpful resources on these topics can be located here - Learning and Understanding Perl and here - Introduction to the Linux Operating System
Creating your Email Server
If this is not already configured by you then you will need to create your email server. opEvents requires an email server in order to sent notifications via email. This email server is configured in /usr/local/omk/conf/opCommon.nmis
. If you do not want to edit text files to configure this, it can be accomplished within the NMIS GUI as well. NMIS will symlink Email, Contacts, etc. already created to opEvents, keep this in mind throughout the guide and choose what works best for you.
- Follow the Wiki guide and set up your Email server
a. Information on setting up an Email server can be found here - Create Email Server
b. Alternative Email configuration through NMIS resource - NMIS8 Email
Configure Contacts
A 'contact' is a variable that can represent one or many email addresses. opEvents can utilize the NMIS contacts file (/usr/local/nmis8/conf/Contact.nmis) or the OMK contacts (/usr/local/omk/conf/Contacts.nmis) file. In order to determine which file your system is using look in opCommon.nmis; find the opevents section then look for the opevents_contacts attribute. Alternatively, you can configure contacts within the NMIS GUI shown below.
- Configure in opEvents by following the Wiki guide to assist in setting up your Contacts - Configure Contacts
2. Alternatively, you can create Contacts within the NMIS GUI.
a. Navigate to NMIS menu -> Setup -> Contact Setup
b. Click 'add' next to Action > add
c. Enter relevant information, click Add at the bottom to confirm.
Creating your Policy and Action
The policy consists of any number of nested if-this-then-that clauses, which specify the conditions an event must conform to and what actions to take in case of a match. Further configuration sections specific to particular actions can be present in the same file. Create a policy that ensures that the engineers you want to alert by email, text, etc. are only alerted by the severity of the event that makes sense within your organization. Doing this reduces event noise allowing your team to quickly identify the important events. Use the examples within the links below as well as the default policies within the EventActions.json (EventActions.nmis for NMIS 8) file to help create a policy right for your environment.
- Follow the basic example of creating a new Escalation Policy - Escalation Policy example
- More detailed Escalation Policy examples and information - Event Actions and Escalation
3. Escalation Policies can also be created within NMIS:
a. Navigate to the NMIS menu -> System -> System Configuration -> Escalation Policy
b. Click 'add' next to Action > add
c. Enter the relevant information and click Add at the bottom to save.
To learn how to add regex to the 'Event Element' field, please refer to the following page: https://docs.community.firstwave.com/wiki/x/MADov
Test your changes
The opevents-cli.exe utility found in /usr/local/omk/bin may be used to create a test event. Create an event that will match a previously configured action rule. For example:
[root@opmantek conf]# /usr/local/omk/bin/opevents-cli.exe act=create-event node=testNode4 stateful=node state=down event="Node DOWN" priority=5 5997bb8cce2c2e6d9453c101
This command will return an event ID. Go to the event context page for this event. In the Actions section of the page there should be an entry stating an email was successfully sent. Keep in mind that this entry will not be present until the flap window and escalation timers have elapsed.