opEvents Events Aggregate

Owned by James Greenwood
Last updated: Aug 12, 2020 by Keith Sinclair (Deactivated)
3 min read

One of the big benefits of opEvents is the big data approach to event management, this allows opEvents to perform sophisticated aggregations which combine data from multiple sources into a new data set which the application can use as needed.  Millions of records can be queried, aggregated, new fields created in milliseconds.

MongoDB aggregations are not for the faint hearted, if you have had some experience with SQL Group by queries with joins, you will have no problem, and you don't need to use all the features to achieve the results you need.

You can learn more about MongoDB Aggregations Here

To help accessing MongoDB many of the Opmantek developers use Robo3T from RoboMongo, an Open Source GUI for MongoDB


Example opEvents Aggregations

Aggregation query for events to summarise the nodes by current state.

db.getCollection('nodes').aggregate([
  //get all nodes from the group HQDEV
  {
    $match: { group: "HQDev" }
  },
  //Get all the states for the matched nodes
  {
    $lookup: {
      from: "state",
      localField: "_id",
      foreignField: "node",
      as: "states"
    }
  },
  //unwinds the states array creating a document per state with node config data
  {
    $unwind: {
      path: "$states"
    }
  },
  //We only want open states
  {
    $match: { 'states.state': { $eq: 'open' } }
  },
  //Join with the event which created this state, we will need this later for the priority 
  {
    $lookup: {
      from: "events",
      localField: "states.eventid_down",
      foreignField: "_id",
      as: "_event"
    }
  },
  {
     $unwind: {
        path: "$_event"
      }
    },
  // group by the node, and accumlate its states
  {
    $group: {
      _id: "$_id",
      states: { $push: { state: "$states.state", stateful: "$states.stateful", priority: "$_event.priority" } }
    }
  },

])


Aggregation query for events to summarise the events by the priority.

Takes all events after a certain time and which are not acknowledged

Firstly it groups by the event name and priority, and counts the amount of events

db.getCollection('events').aggregate([
  {
    $match: { "time": { "$gte": 1577836800 }, "acknowledged": { "$eq": 0 } }
  },
  {
    $group: {
      _id: { event: "$event", priority: "$priority" },
      priorities: { $push: "$priority" },
      total: { $sum: 1 }
    }
  },
  {
    $group: {
      _id: "$_id.event",
      priorities: { $push: { priority: "$_id.priority", count: "$total" } },
    }
  }
])

Priority sorted for all events which are not acknowledged

db.getCollection('events').aggregate([
  {
    $match: { "time": { "$gte": 1577836800 }, "acknowledged": { "$eq": 0 } }
  },
  {
    $group: {
      _id: "$priority",
      total: { $sum: 1 }
    }
  },
  {$sort: {"_id": 1}}

])