Versions Compared

Key

  • This line was added.
  • This line was removed.
  • Formatting was changed.

//One of the big benefits of opEvents is the big data approach to event management, this allows opEvents to perform sophisticated aggregations which combine data from multiple sources into a new data set which the application can use as needed.  Millions of records can be queried, aggregated, new fields created in milliseconds.

MongoDB aggregations are not for the faint hearted, if you have had some experience with SQL Group by queries with joins, you will have no problem, and you don't need to use all the features to achieve the results you need.

You can learn more about MongoDB Aggregations Here

To help accessing MongoDB many of the Opmantek developers use Robo3T from RoboMongo, an Open Source GUI for MongoDB


Table of Contents

Example opEvents Aggregations

Aggregation query for events to summarise the nodes by current state.

Code Block
languagejs
db.getCollection('nodes').aggregate([
 	 //get all nodes from the group HQDEV
    {
      $match: { group: "HQDev" }
  },
   },
//Get all the states for the matched nodes

   {
   
    $lookup: {
     
      from: "state",

           localField: "_id",
 
          foreignField: "node",
     
      as: "states"
     }
  },
    }, //unwinds the states array creating a document per state with node config data
 
  {
   
    $unwind: {
  
         path: "$states"
     }
  }
,
   },
//We only want open states
    {

       $match: { 'states.state': { $eq: 'open' } }
  },
 }, //Join groupwith by the node,event thewhich statecreated andthis thestate, statefulwe elementwill need this later for {the priority 
  {
   $group $lookup: {
     _id from: {node "events",
      localField: "$states.eventid_iddown",
state      foreignField: "$states.state_id",
stateful      as: "$states.stateful_event"
    }
  },
  {
     total$unwind: {$sum: 1}
        path: "$_event"
      }
    },
  //Then group by the node, and collectingaccumlate all theits states for the node

   {
        $group : {
      _id : "$_id.node",
      states: { $push: { state: "$_id$states.state", stateful: "$_id$states.stateful", countpriority: "$total$_event.priority" } }
        }

   },

  
    ])

Image RemovedImage Added


Aggregation query for events to summarise the events by the priority.

Takes all events after a certain time and which are not acknowledged

...

Code Block
languagejs
db.getCollection('events').aggregate([
  {
    $match: { "time": { "$gte": 1577836800 }, "acknowledged": { "$eq": 0 } }
  },
  {
    $group: {
      _id: { event: "$event", priority: "$priority" },
      priorities: { $push: "$priority" },
      total: { $sum: 1 }
    }
  },
  {
    $group: {
      _id: "$_id.event",
      priorities: { $push: { priority: "$_id.priority", count: "$total" } },
    }
  }
])



Priority sorted for all events which are not acknowledged

Code Block
languagejs
db.getCollection('events').aggregate([
  {
    $match: { "time": { "$gte": 1577836800 }, "acknowledged": { "$eq": 0 } }
  },
  {
    $group: {
      _id: "$priority",
      total: { $sum: 1 }
    }
  },
  {$sort: {"_id": 1}}

])

...