Table of Contents |
---|
Overview
Dealing with NetFlow traffic analysis commonly involves large to huge amounts of data, which poses quite a few scalability challenges. Meeting these requires certain trade-offs to be made by any NetFlow analyzer, not just opFlow. This page documents the configuration options and behaviours related to making opFlow scale well.
...
You can define summary stages in the configuration file, under opflow_summary_stages
; a stage definition requires a name (allowed characters A-Z, a-z, 0-9, _ and -), and a period
(in seconds). The summarised data will be stored in a collection named summary_
<stagename>. You can optionally set up database capping for this collection (with the collection_size
property, in bytes), and collapsing of unimportant conversations (with the collapse_min_bytes
and/or collapse_min_pkts
settings - zero or not set disables collapsing, and collapsing happens if either of the two criteria is met).
Extended Summarisation
Since opFlow 3.0 and all opFlowSP versions, you are able to configure further summarised flows in order to be able to keep more historical information. In opFlow you will notice that the installer comes with the following extra summary options:
Code Block | ||
---|---|---|
| ||
'opflow_summary_stages' => {
'daily' => {
'collapse_min_bytes' => 102400,
'collapse_min_pkts' => 128,
'collection_size' => 1073741824,
'period' => 3600
},
'hourly' => {
collapse_min_bytes => 1024,
collapse_min_pkts => 5,
collection_size => 1073741824, # 1gb
period => 3600,
},
'quarterhr' => {
'collapse_min_bytes' => 1024,
'collapse_min_pkts' => 5,
'collection_size' => 1073741824,
'period' => 900
}
}, |
Currently opFlowSP has the a single summary option, but you can easily add more:
Code Block | ||
---|---|---|
| ||
'opflow_summary_stages' => {
'quarterhr' => {
'collapse_min_bytes' => 1024,
'collapse_min_pkts' => 5,
'collection_size' => 1073741824,
'period' => 900
}
}, |
configuration | Unit | Description |
---|---|---|
collapse_min_bytes | bytes | Collapse all conversations with less than X bytes of traffic in the whole period into one; set to zero/undef to disable |
collapse_min_pkts | packets | or window by minimum number of packets. a match of either criterion will cause collapsing |
collection_size | bytes | Collection capped size |
period | seconds | Summary period |
Changing Summarisations
You can remove an existing summarisation from the configuration and restart the opflow daemon, you will need to drop the database collection manually once you are sure.
When adding additional summarisation intervals you will need to run the DB setup command as it will setup collection capping. You will want to stop the opflow daemon as well as omkd then restart them once you have made the changes you wanted.
Code Block |
---|
service opflowd stop
opflow-cli.exe act=setup-db
service opflowd start |
Also, and this is worth noting, the daemon will create all the summaries when it starts, so depending on how much data is already present, this may increase the load on the server.
opFlow GUI Modes
The opFlow GUI in version 3 includes two different dashboard pages, one optimised for high-volume and one optimised for high precision.
...
In this mode the dashboard shows the data summarised in three different ways, Top Talkers, Top Applications and Top Applications plus Sources, again sorted by traffic volume in bytes. Again, the Advanced menu lets you select the sort field (Summary Field), changing Summary Type changes only the Flows over Time chart in this mode..