This guide steps through the changes to settings and filesystem privileges required to enhance the security of on a standalone machine running NMIS9 Modules on Ubuntu Linux.

This guide has also been tested on RHEL8.8, the steps are exactly the same, and along with verification. The only difference is that you have to use service instead of systemctl on RHEL/Centos distributions.

Info

This guide runs top to bottom as a sequence. It is important that this sequence is followed in order to prevent issues on restart.

Backups are highly recommended.

Table of Contents

Preparation

The changes involved in this document include the use of recursive rights changes and recursive directory deletion as the "root" user.

...

Info
DO NOT PROCEED WITHOUT BACKUPS

Tested Modules

Be aware that in the event the system is running any other applications/modules that require access to the file structures on the following list there is a very high probability that their rights will be blocked and the additional processes could fail to execute properly.

nmis9
opAddress
opAdmin
opCharts
opConfig
opEvents
opHA
opReports

System Snapshot/Backup

For System Snapshots and Backups you will need to refer to the documentation for your environment.

...

Full system backups, including Operating System and Data - Contact your IT department, Vendor Support, or Integrator
Virtual Machine Snapshots - Refer to the solution guides for your environment

User Setup

Info
All commands in this document are run as the root user unless otherwise stated.

...

The following command will not work if there was ANY issue error issued when creating the user or group. If you get an error in creating the user this needs to be resolved before proceeding.

Adjust User Groups

Next we will add the USER "nmis" to the GROUP "omkadmin" and the add USER "omkadmin" to the GROUP "nmis"

...

Code Block
root@server:~# groups nmis nmis adm plugdev netdev lxd google-sudoers omkadmin root@server:~# groups omkadmin omkadmin nmis

Shutdown all impacting services

...

Open-AudIT and the Apache user

If you are using Open-AudIT, you will also need to add the Apache user (www-data) to the omkadmin group.

Code Block
usermod -a -G omkadmin www-data

Shutdown all impacting services

This section will bring all services that are going to be altered down this will include; crond, nmis9d, omk daemons.

Because we are shutting crond down some system administrative tasks may not fire during this period.

...

When processes have all shutdown gracefully the previous command will return something very similar to the following. The first number on the line can be ignored:

Code Block
root@server:~# ps ax \| grep -E "(nmis\|cron\| op)" 1443224 pts/0 S+ 0:00 grep --color=auto -E (nmis\|cron\| op) root@server:~#

Ownership and Rights Changes

The following set of commands will run through and alter all of the file and directory rights as the key security task. You should be able to cut and past the entire block into the cli as root user.

...

Code Block

linenumbers	true

root@server:~# ls -la /usr/local/omk/var/lib/common/
total 20
drwxrws--T  5 omkadmin omkadmin 4096 Jul 13 11:55 .
drwxrwx---  3 omkadmin omkadmin 4096 Jun 28 03:41 ..
drwx--S---  3 nmis     omkadmin 4096 Jul 13 11:55 par-6e6d6973
drwx--S---  3 omkadmin omkadmin 4096 Jul 13 11:55 par-6f6d6b61646d696e
drwx--S--- 12 root     omkadmin 4096 Jul 13 12:57 par-726f6f74

Change Process PID locations:

In /usr/local/omk/conf/opCommon.json set each *_pid entry as follows (note that each service has its own pid directory under /var/run):

...

Code Block

root@server:/run# grep "/var/run/omk" /usr/local/omk/conf/opCommon.json
      "opflowd_pid" : "/var/run/omk/opflowd.pid",
      "opchartsd_pid" : "/var/run/omk/opchartsd.pid",
      "opeventsd_pid" : "/var/run/omk/opeventsd.pid",
      "opconfigd_pid" : "/var/run/omk/opconfigd.pid",
root@server:/run#

SYSTEMCTL Service Files

Locate necessary service files for alteration using a grep command

...

omkd.service
opchartsd.service
opconfigd.service
opeventsd.service

omkd.service

Code Block

sed -i 's/\[Service\]/\[Service\]\nUser\=omkadmin\nGroup\=omkadmin/g' /etc/systemd/system/omkd.service
sed -i 's/\/var\/run/\/var\/run\/omk/g' /etc/systemd/system/omkd.service
sed -i 's/\[Service\]/\[Service\]\nPermissionsStartOnly\=true\nExecStartPre\=\/bin\/sh \-c \"mkdir \-p \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chown omkadmin\.omkadmin \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chmod 3700 \/var\/run\/omk\/\"/g' /etc/systemd/system/omkd.service

opchartsd.service

Code Block

sed -i 's/\[Service\]/\[Service\]\nUser\=omkadmin\nGroup\=omkadmin/g' /etc/systemd/system/opchartsd.service
sed -i 's/\/var\/run/\/var\/run\/omk/g' /etc/systemd/system/opchartsd.service
sed -i 's/\[Service\]/\[Service\]\nPermissionsStartOnly\=true\nExecStartPre\=\/bin\/sh \-c \"mkdir \-p \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chown omkadmin\.omkadmin \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chmod 3700 \/var\/run\/omk\/\"/g' /etc/systemd/system/opchartsd.service

opconfigd.service

Code Block

sed -i 's/\[Service\]/\[Service\]\nUser\=omkadmin\nGroup\=omkadmin/g' /etc/systemd/system/opconfigd.service
sed -i 's/\/var\/run/\/var\/run\/omk/g' /etc/systemd/system/opconfigd.service
sed -i 's/\[Service\]/\[Service\]\nPermissionsStartOnly\=true\nExecStartPre\=\/bin\/sh \-c \"mkdir \-p \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chown omkadmin\.omkadmin \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chmod 3700 \/var\/run\/omk\/\"/g' /etc/systemd/system/opconfigd.service

opeventsd.service

Code Block

sed -i 's/\[Service\]/\[Service\]\nUser\=omkadmin\nGroup\=omkadmin/g' /etc/systemd/system/opeventsd.service
sed -i 's/\/var\/run/\/var\/run\/omk/g' /etc/systemd/system/opeventsd.service
sed -i 's/\[Service\]/\[Service\]\nPermissionsStartOnly\=true\nExecStartPre\=\/bin\/sh \-c \"mkdir \-p \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chown omkadmin\.omkadmin \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chmod 3700 \/var\/run\/omk\/\"/g' /etc/systemd/system/opeventsd.service

Confirm SYSTEMCTL Changes:

The output of the above commands will have been to adjust the service files to add some additional entries to the Service section of the configs and to alter PID references in the omkd.service file

...

With the new USER, GROUP and altered PID entries as per the following - post change. You can use this example as a guide for a concise review of each file if you wish.

Code Block

title	Updated Service File - omkd.service

# simple systemd unit file for the Opmantek Webserver daemon

[Unit]
Description=Opmantek Webserver
After=network-online.target
After=mongod.service
Requires=mongod.service
Wants=mongod.service
Wants=network-online.target

[Install]
WantedBy=multi-user.target

[Service]
UserPermissionsStartOnly=omkadmintrue
Group=omkadmin
Type=forking
Restart=no
EnvironmentFile=/etc/environment
ExecStartPre=/bin/sh -c "mkdir -p /var/run/omk/"
ExecStartPre=/bin/sh -c "chown omkadmin.omkadmin /var/run/omk/"
ExecStartPre=/bin/sh -c "chmod 3700 /var/run/omk/"
User=omkadmin
Group=omkadmin
Type=forking
Restart=no
EnvironmentFile=/etc/environment
PIDFile=/var/run/omk/opmantek.exe.pid
TimeoutSec=120s
KillMode=process
ExecStart=/usr/local/omk/script/opmantek.pl -f -p /var/run/omk/opmantek.exe.pid -r

...

Code Block

root@server:/etc/systemd/system# grep -s omk /etc/systemd/system/* | grep -vE "pl$"
/etc/systemd/system/omkd.service:ExecStartPre=/bin/sh -c "mkdir -p /var/run/omk/"
/etc/systemd/system/omkd.service:ExecStartPre=/bin/sh -c "chown omkadmin.omkadmin /var/run/omk/"
/etc/systemd/system/omkd.service:ExecStartPre=/bin/sh -c "chmod 0700 /var/run/omk/"
/etc/systemd/system/omkd.service:User=omkadmin
/etc/systemd/system/omkd.service:Group=omkadmin
/etc/systemd/system/omkd.service:PIDFile=/var/run/omk/opmantek.exe.pid
/etc/systemd/system/omkd.service:ExecStart=/usr/local/omk/script/opmantek.pl -f -p /var/run/omk/opmantek.exe.pid -r
/etc/systemd/system/opchartsd.service:ExecStartPre=/bin/sh -c "mkdir -p /var/run/omk/"
/etc/systemd/system/opchartsd.service:ExecStartPre=/bin/sh -c "chown omkadmin.omkadmin /var/run/omk/"
/etc/systemd/system/opchartsd.service:ExecStartPre=/bin/sh -c "chmod 0700 /var/run/omk/"
/etc/systemd/system/opchartsd.service:User=omkadmin
/etc/systemd/system/opchartsd.service:Group=omkadmin
/etc/systemd/system/opchartsd.service:PIDFile=/var/run/omk/opchartsd.pid
/etc/systemd/system/opconfigd.service:ExecStartPre=/bin/sh -c "mkdir -p /var/run/omk/"
/etc/systemd/system/opconfigd.service:ExecStartPre=/bin/sh -c "chown omkadmin.omkadmin /var/run/omk/"
/etc/systemd/system/opconfigd.service:ExecStartPre=/bin/sh -c "chmod 0700 /var/run/omk/"
/etc/systemd/system/opconfigd.service:User=omkadmin
/etc/systemd/system/opconfigd.service:Group=omkadmin
/etc/systemd/system/opconfigd.service:PIDFile=/var/run/omk/opconfigd.pid
/etc/systemd/system/opeventsd.service:ExecStartPre=/bin/sh -c "mkdir -p /var/run/omk/"
/etc/systemd/system/opeventsd.service:ExecStartPre=/bin/sh -c "chown omkadmin.omkadmin /var/run/omk/"
/etc/systemd/system/opeventsd.service:ExecStartPre=/bin/sh -c "chmod 0700 /var/run/omk/"
/etc/systemd/system/opeventsd.service:User=omkadmin
/etc/systemd/system/opeventsd.service:Group=omkadmin
/etc/systemd/system/opeventsd.service:PIDFile=/var/run/omk/opeventsd.pid

Logrotate Changes

/etc/logrotate.d/omk-rotate.conf needs to be edited and each section of the logrotate script needs to include:

...

The su directive is new and is required so that logrotate will continue to work when writing to a no-root owned file (non-privileged). We place it at after the "endscript" statement.

CRON Job Changes

The following commands are used to change all of the default cronjobs. Replacing all "root" user entries with "omkadmin" user entries.

Code Block

sed -i 's/   root\t/\tomkadmin\t/g' opaddress
sed -i 's/\troot\t/\tomkadmin\t/g' opconfig
sed -i 's/\troot\t/\tomkadmin\t/g' opevents
sed -i 's/\troot\t/\tomkadmin\t/g' opha
sed -i 's/\troot\t/\tomkadmin\t/g' oplicense
sed -i 's/\troot\t/\tomkadmin\t/g' opreports

Rights Checks

If crond was not disabled prior to updating all of the directory rights this list will likely return many files owned by root.root. Those specific files will need to be manually changed to omkadmin.omkadmin ownership.

Code Block

sudo find /usr/local/omk/ ! -group omkadmin ! -regex '/usr/local/omk/var/lib/common/par-.+' -exec ls -lAhd '{}' \;;
-rw-rw-r-- 1 nmis nmis 453 Jul 13 20:15 /usr/local/omk/var/opcore/registry/open-audit.json
-rw-rw-r-- 1 nmis nmis 3.5K Jul 13 20:15 /usr/local/omk/var/opcore/registry/opEvents.json
-rw-rw-r-- 1 nmis nmis 1.5K Jul 13 20:15 /usr/local/omk/var/opcore/registry/opHA.json
-rw-rw-r-- 1 nmis nmis 5.1K Jul 13 20:15 /usr/local/omk/var/opcore/registry/opCharts.json
-rw-rw-r-- 1 nmis nmis 2.0K Jul 13 20:15 /usr/local/omk/var/opcore/registry/opConfig.json
-rw-rw-r-- 1 nmis nmis 108 Jul 13 20:27 /usr/local/omk/var/opevents/file_state/_usr_local_nmis9_logs_event.log.json

Rights Check CronJob

Create cronjob /etc/cron.d/omk_check_omkadmin_user_group:

...

Code Block

touch /etc/cron.d/omk_check_omkadmin_user_group
echo "PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin" >> /etc/cron.d/omk_check_omkadmin_user_group
echo "# m h dom mon dow user command" >> /etc/cron.d/omk_check_omkadmin_user_group
echo "40 * * * * root find /usr/local/omk ! -group omkadmin ! -group nmis ! -regex '/usr/local/omk/var/lib/common/par-.+' -exec ls -lAhd '{}' \;;" >> /etc/cron.d/omk_check_omkadmin_user_group
echo "42 * * * * omkadmin find /usr/local/omk ! -writable -exec ls -lAhd '{}' \;;" >> /etc/cron.d/omk_check_omkadmin_user_group
echo "44 * * * * root find /usr/local/omk -perm /+2000 -exec ls -lAhd '{}' \;;" >> /etc/cron.d/omk_check_omkadmin_user_group
echo "46 * * * * root find /usr/local/omk -perm /+4000 -exec ls -lAhd '{}' \;;" >> /etc/cron.d/omk_check_omkadmin_user_group

Reboot the machine

Rebooting the machine will restore all services and allow for system maintenance and startup tasks that were halted to be run as required with the new users and privileges.

Code Block
sudo reboot

Full Script

Info
Note that the following script has been added for convenience at this stage and has no checks or error handling embedded.

...

Code Block

# Setup User Logins and Groups
useradd -m -U omkadmin
passwd -l omkadmin
usermod -a -G omkadmin nmis
usermod -a -G nmis omkadmin

# ShowdownNOTE all- impacting/impacteduncomment services
/usr/local/omk/bin/checkomkdaemons.sh stop
systemctl stop the below if also using Open-AudIT
# usermod -a -G omkadmin www-data

# Showdown all impacting/impacted services
/usr/local/omk/bin/checkomkdaemons.sh stop
systemctl stop nmis9d
systemctl stop cron
sleep 10
systemctl stop nmis9d

# START of standard installer changes
OMK_DIR=/usr/local/omk

# 
echo Set OMK directory structure writable by group:
sudo chown -R omkadmin:omkadmin "${OMK_DIR}";
sudo find "${OMK_DIR}" -type d -exec chmod 0770 '{}' \;;

# 
echo Set user and group able to write files:
sudo find "${OMK_DIR}" -type f -exec chmod 0660 '{}' \;;

# 
echo Set scripts executable by user and group:
sudo find "${OMK_DIR}/script" -type f -exec chmod 0770 '{}' \;;

# 
echo Set scripts executable by user and group:
sudo find "${OMK_DIR}/bin" -type f -exec chmod 0770 '{}' \;;
 
# END of standard installer changes

#
echo Delete existing PAR subdirectories as we may have set incorrect permissions on this directory
sudo rm -Rf ${PAR_GLOBAL_TMPDIR}/par-*
sudo rm -Rf /tmp/par-*

#
echo Set sticky bit on $PAR_GLOBAL_TMPDIR directory and only executable by root.
sudo chmod 1700 ${PAR_GLOBAL_TMPDIR}

#
echo Recreate $PAR_GLOBAL_TMPDIR/par- directories for root,nmis and omkadmin
sudo ${OMK_DIR}/bin/patch_config.exe 2> /dev/null
sudo -u nmis ${OMK_DIR}/bin/patch_config.exe 2> /dev/null
sudo -u omkadmin ${OMK_DIR}/bin/patch_config.exe 2> /dev/null

#
echo Update opCommon.json config with new PID directories
sed -i 's/var\/run/var\/run\/omk/g' /usr/local/omk/conf/opCommon.json

#
echo Update SYSTEMCTL Server Files
#
echo omkd.service
sed -i 's/\[Service\]/\[Service\]\nUser\=omkadmin\nGroup\=omkadmin/g' /etc/systemd/system/omkd.service
sed -i 's/\/var\/run/\/var\/run\/omk/g' /etc/systemd/system/omkd.service
sed -i 's/\[Service\]/\[Service\]\nPermissionsStartOnly\=true\nExecStartPre\=\/bin\/sh \-c \"mkdir \-p \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chown omkadmin\.omkadmin \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chmod 3700 \/var\/run\/omk\/\"/g' /etc/systemd/system/omkd.service

#
echo opchartsd.service
sed -i 's/\[Service\]/\[Service\]\nUser\=omkadmin\nGroup\=omkadmin/g' /etc/systemd/system/opchartsd.service
sed -i 's/\/var\/run/\/var\/run\/omk/g' /etc/systemd/system/opchartsd.service
sed -i 's/\[Service\]/\[Service\]\nPermissionsStartOnly\=true\nExecStartPre\=\/bin\/sh \-c \"mkdir \-p \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chown omkadmin\.omkadmin \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chmod 3700 \/var\/run\/omk\/\"/g' /etc/systemd/system/opchartsd.service

#
echo opconfigd.service
sed -i 's/\[Service\]/\[Service\]\nUser\=omkadmin\nGroup\=omkadmin/g' /etc/systemd/system/opconfigd.service
sed -i 's/\/var\/run/\/var\/run\/omk/g' /etc/systemd/system/opconfigd.service
sed -i 's/\[Service\]/\[Service\]\nPermissionsStartOnly\=true\nExecStartPre\=\/bin\/sh \-c \"mkdir \-p \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chown omkadmin\.omkadmin \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chmod 3700 \/var\/run\/omk\/\"/g' /etc/systemd/system/opconfigd.service

#
echo opeventsd.service
sed -i 's/\[Service\]/\[Service\]\nUser\=omkadmin\nGroup\=omkadmin/g' /etc/systemd/system/opeventsd.service
sed -i 's/\/var\/run/\/var\/run\/omk/g' /etc/systemd/system/opeventsd.service
sed -i 's/\[Service\]/\[Service\]\nPermissionsStartOnly\=true\nExecStartPre\=\/bin\/sh \-c \"mkdir \-p \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chown omkadmin\.omkadmin \/var\/run\/omk\/\"\nExecStartPre\=\/bin\/sh \-c \"chmod 3700 \/var\/run\/omk\/\"/g' /etc/systemd/system/opeventsd.service

#
echo Update logrotate config
sed -i 's/create 0660 nmis nmis/create 0660 omkadmin omkadmin/g' /etc/logrotate.d/omk-rotate.conf
sed -i 's/endscript/endscript\n\tsu omkadmin omkadmin/g' /etc/logrotate.d/omk-rotate.conf

#
echo Update all crontab job owners
sed -i 's/   root\t/\tomkadmin\t/g' /etc/cron.d/opaddress
sed -i 's/\troot\t/\tomkadmin\t/g' /etc/cron.d/opconfig
sed -i 's/\troot\t/\tomkadmin\t/g' /etc/cron.d/opevents
sed -i 's/\troot\t/\tomkadmin\t/g' /etc/cron.d/opha
sed -i 's/\troot\t/\tomkadmin\t/g' /etc/cron.d/oplicense
sed -i 's/\troot\t/\tomkadmin\t/g' /etc/cron.d/opreports

#
echo Add an Hourly Rights Check to CRONTAB
touch /etc/cron.d/omk_check_omkadmin_user_group
echo "PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin" >> /etc/cron.d/omk_check_omkadmin_user_group
echo "# m h dom mon dow user command" >> /etc/cron.d/omk_check_omkadmin_user_group
echo "40 * * * * root find /usr/local/omk ! -group omkadmin ! -group nmis ! -regex '/usr/local/omk/var/lib/common/par-.+' -exec ls -lAhd '{}' \;;" >> /etc/cron.d/omk_check_omkadmin_user_group
echo "42 * * * * omkadmin find /usr/local/omk ! -writable -exec ls -lAhd '{}' \;;" >> /etc/cron.d/omk_check_omkadmin_user_group
echo "44 * * * * root find /usr/local/omk -perm /+2000 -exec ls -lAhd '{}' \;;" >> /etc/cron.d/omk_check_omkadmin_user_group
echo "46 * * * * root find /usr/local/omk -perm /+4000 -exec ls -lAhd '{}' \;;" >> /etc/cron.d/omk_check_omkadmin_user_group

Hardening NMIS

The above guide can also be applied to the nmis user and change the nmid9d service to execute and own all of it's process's rather than having the root process own the nmis workers.

This can be done concurrently, independantly or after following the above guide.

This has been tested on Ubuntu 20.04/RHEL 8.8 installations running default service files and directory settings for nmis and omk modules from their respective installers. You will need to modify some commands into their respective RHEL/Centos counterparts in the above script and steps.

Info
DO NOT PROCEED WITHOUT BACKUPS BEFORE MODIFYING NMIS9D.SERVICE

Shutdown all impacting services

Code Block
/usr/local/omk/bin/checkomkdaemons.sh stop systemctl stop nmis9d systemctl stop cron

SYSTEMCTL Service File changes for NMIS

Code Block
sudo sed -i '/\[Service\]/a User=nmis\nGroup=nmis' /etc/systemd/system/nmis9d.service

CRON Job Changes for NMIS

Code Block
sed -i 's/\troot/\/nmis\t/g' /etc/cron.d/nmis9

Reboot the machine

Rebooting the machine will restore all services and allow for system maintenance and startup tasks that were halted to be run as required with the new users and privileges.

Code Block
sudo reboot

Verify the changes

Image Added

Versions Compared

Old Version 1

New Version Current

Key

Preparation

Tested Modules

System Snapshot/Backup

User Setup

Adjust User Groups

Shutdown all impacting services

Open-AudIT and the Apache user

Shutdown all impacting services

Ownership and Rights Changes

Change Process PID locations:

SYSTEMCTL Service Files

omkd.service

opchartsd.service

opconfigd.service

opeventsd.service

Confirm SYSTEMCTL Changes:

Logrotate Changes

CRON Job Changes

Rights Checks

Rights Check CronJob

Reboot the machine

Full Script

Hardening NMIS

SYSTEMCTL Service File changes for NMIS

CRON Job Changes for NMIS

Reboot the machine

Verify the changes

Page Comparison

Versions Compared

Old Version 1

New Version Current

Key

Preparation

Tested Modules

System Snapshot/Backup

User Setup

Adjust User Groups

Shutdown all impacting services

Open-AudIT and the Apache user

Shutdown all impacting services

Ownership and Rights Changes

Change Process PID locations:

SYSTEMCTL Service Files

omkd.service

opchartsd.service

opconfigd.service

opeventsd.service

Confirm SYSTEMCTL Changes:

Logrotate Changes

CRON Job Changes

Rights Checks

Rights Check CronJob

Reboot the machine

Full Script

Hardening NMIS

SYSTEMCTL Service File changes for NMIS

CRON Job Changes for NMIS

Reboot the machine

Verify the changes