...
- What devices, and operating systems will be sending logs.
- What applications will be sending logs.
- What protocol will be used to send log messages.
- What software will be used to send the log messages.What timezone are each of the devices sending logs in.
- What criticality of logs is required.
...
TCP should where reliable logging is required, UDP works very well, 99.99% of the time.
Logging Severity Levels
The requirement is to send level 6 and above.
Value | Severity | Keyword |
---|---|---|
0 | Emergency | emerg |
1 | Alert | alert |
2 | Critical | crit |
3 | Error | err |
4 | Warning | warn |
5 | Notice | notice |
6 | Informational | info |
7 | Debug | debug |
Full details for syslog severity levels https://en.wikipedia.org/wiki/Syslog#Severity_level
Handy cross reference: opEvents priority levels vs. NMIS and Syslog levels
Time Handling
There are two primary concerns about time, one clock drift the second is multiple timezones.
There are several options for handling timezone, the following are considered best practices for IT management in this regard:
- All devices should use NTP to ensure accurate time is set
- All devices should have their timezone set to their localtime or to a common timezone, e.g. UTC
- All devices should include the timezone when timestamping, ideally as an offset, e.g. +00
However it is often difficult to get all devices especially devices already installed to do all these things, so a great option is to make the logging server the authoritive time, NTP is setup a timezone is selected, and all logs received by it will be stamped with the time when the log is received.
Centralised Logging and Archiving Solution
...
Source | Method and Transport |
---|---|
Windows 2003 Servers | nxlog monitoring Windows Event log, transport over syslog |
Windows 2008 Servers | nxlog monitoring Windows Event log, transport over syslog |
Windows 2012 Servers | nxlog monitoring Windows Event log, transport over syslog |
CentOS Linux 5.x | rsyslog 3.x |
CentOS Linux 6.x | rsyslog 7.6 |
Cisco IOS Switches | Native IOS syslog |
Cisco IOS Routers | Native IOS syslog |
Monkey Auth System | nxlog running on Windows. |
Elephant Financials | rsyslog running on Linux |
Logging Severity Levels
...
https://en.wikipedia.org/wiki/Syslog#Severity_level
Value | Severity |
---|---|
0 | Emergency |
1 | Alert |
2 | Critical |
3 | Error |
4 | Warning |
5 | Notice |
6 | Informational |
7 | Debug |
syslog Facility
The best reference is: https://en.wikipedia.org/wiki/Syslog#Facility
...
Device Type | syslog facility | Log file |
---|---|---|
| local0 | /data/log/local0.log |
Log server to log server (future) | local1 | /data/log/local1.log |
Application logging e.g. MonkeyAuth | local2 | /data/log/local2.log |
Windows servers (nxlog default) | local3 | /data/log/local3.log |
Cisco ASA default (VMware ESXi default) | local4 | /data/log/local4.log |
| local5 | /data/log/local5.log |
Linux syslog | local6 | /data/log/local6.log |
Cisco Routers and Switches | local7 | /data/log/local7.log |
Alternate file naming can be supported if required, e.g. cisco.log instead of local7.log.
Centralised Logging Implementation
Central rsyslog Server Configuration
Translating all the above into the configuration the following are the most important parts.
It should be noted that trying to use the /etc/rsyslog.d scheme did not work.
Enable rsyslog to receive UDP and TCP syslogs
By default (to prevent DOS) rsyslog is configured to not receive syslogs from remote servers.
Code Block |
---|
# Provides UDP syslog reception
$ModLoad imudp.so
$UDPServerRun 514
# Provides TCP syslog reception
$ModLoad imtcp.so
$InputTCPServerRun 514 |
Configure each Facility to be saved into Files
Based on the table above the following would be the configuration
Code Block |
---|
local0.* /usr/local/nmis8/logs/local0.log
local1.* /usr/local/nmis8/logs/local1.log
local2.* /usr/local/nmis8/logs/local2.log
local3.* /usr/local/nmis8/logs/local3.log
local4.* /usr/local/nmis8/logs/local4.log
local5.* /usr/local/nmis8/logs/local5.log
local6.* /usr/local/nmis8/logs/local6.log
local7.* /usr/local/nmis8/logs/local7.log |
Handling Different Times and Time Zones
The following configuration shows how to create an rsyslog template and apply that to the logs being received, this example also adds high precision time, which is supported by opEvents.
Code Block |
---|
$template ServerTime,"%timegenerated%.%timegenerated:::date-subseconds% %HOSTNAME% %syslogtag%%msg%\n"
local5.* /usr/local/nmis8/logs/local5.log;ServerTime |
All syslog received to the facility local5 will be timestamped with the receiving syslog servers high precision time.
Sample Configuration for rsyslog 7.6
...