...
Code Block |
---|
ls -lAth /usr/local/omk/var/lib/common/ drwx--S---. 3 omkadmin omkadmin 4.0K Jan 22 08:54 par-6f6d6b61646d696e drwx--S---. 9 root omkadmin 4.0K Jan 22 08:52 par-726f6f74 drwx--S---. 3 nmis omkadmin 4.0K Jan 22 08:50 par-6e6d6973 # and /tmp/ is okay too: sudo ls -lAth /tmp/par-* ls: cannot access /tmp/par-*: No such file or directory |
Services
At this stage we have services running as root, relying on root privileges to access directory structures with user and group being omkadmin. This works, but isn't ideal, so I attempted to get the systemd services running as user and group omkadmin and with minor changes got the services started with omkadmin as user and group:
Code Block |
---|
ps aux|grep omkadmin|grep -v grep|grep -v ps;
root 8542 0.0 0.0 156852 5668 ? Ss 17:10 0:00 sshd: omkadmin [priv]
omkadmin 8545 0.0 0.0 156852 2596 ? S 17:10 0:00 sshd: omkadmin@pts/0
omkadmin 8546 0.0 0.0 116308 2976 pts/0 Ss 17:10 0:00 -bash
omkadmin 16596 0.5 3.5 744052 282988 ? Ss 18:16 0:02 opmantek.pl-webserver -f -p /var/run
/omkd/opmantek.exe.pid -r
omkadmin 16674 0.0 3.6 750236 291808 ? S 18:17 0:00 opmantek.pl-webserver -f -p /var/run
/omkd/opmantek.exe.pid -r
omkadmin 16675 0.0 3.6 749948 291368 ? S 18:17 0:00 opmantek.pl-webserver -f -p /var/run
/omkd/opmantek.exe.pid -r
omkadmin 16676 0.0 3.6 749100 290476 ? S 18:17 0:00 opmantek.pl-webserver -f -p /var/run
/omkd/opmantek.exe.pid -r
omkadmin 16677 0.0 3.6 749104 290464 ? S 18:17 0:00 opmantek.pl-webserver -f -p /var/run
/omkd/opmantek.exe.pid -r
omkadmin 16678 0.0 3.6 749048 290520 ? S 18:17 0:00 opmantek.pl-webserver -f -p /var/run
/omkd/opmantek.exe.pid -r
omkadmin 16734 0.0 2.0 516592 162804 ? Ss 18:17 0:00
opconfigd
omkadmin 16735 2.0 2.2 533916 178636 ? S 18:17 0:07 opconfigd
worker
omkadmin 16768 0.0 1.7 397504 136556 ? Ss 18:18 0:00
opeventsd
omkadmin 16769 7.8 2.2 529444 178300 ? S 18:18 0:24 opeventsd.
worker
omkadmin 16790 0.0 2.1 528736 174528 ? S 18:18 0:00 opeventsd.tail.tivoli.
log
omkadmin 16791 1.8 2.1 528868 175092 ? S 18:18 0:05 opeventsd.tail.event.
log
omkadmin 16792 0.0 2.1 528736 174528 ? S 18:18 0:00 opeventsd.tail.trap.
log
omkadmin 16794 0.0 2.1 528736 174524 ? S 18:18 0:00 opeventsd.tail.cisco.
log
omkadmin 16796 0.0 2.1 528736 174528 ? S 18:18 0:00 opeventsd.tail.winlogd.
log
omkadmin 16841 0.0 1.4 339892 112900 ? Ss 18:18 0:00
opchartsd
omkadmin 16842 0.1 1.9 505236 157140 ? S 18:18 0:00 opchartsd worker |
The changes needed to get this working are needed to overcome the fact that normal users, such as omkadmin, don't have write access to the /var/run/ directory for their pids:
In /usr/local/omk/conf/opCommon.json set each *_pid entry as follows (note that each service has its own pid directory under /var/run):
- "opchartsd_pid" : "/var/run/opchartsd/opchartsd.pid"
- "opconfigd_pid" : "/var/run/opconfigd/opconfigd.pid"
- "opeventsd_pid" : "/var/run/opeventsd/opeventsd.pid"
Info |
---|
Note: that omkd service sets its pid directory as a parameter in the command that starts the service |
Each service file needs slight adjustments mostly to do with each service having its own pid directory, for example for opchartsd.service under [service]:
Code Block |
---|
User=omkadmin
Group=omkadmin
PIDFile=/var/run/opchartsd/opchartsd.pid
PermissionsStartOnly=true
ExecStartPre=/bin/sh -c "mkdir -p /var/run/opchartsd/"
ExecStartPre=/bin/sh -c "chown omkadmin:omkadmin /var/run/opchartsd/"
ExecStartPre=/bin/sh -c "chmod 0700 /var/run/opchartsd/" |
'User=' line is a new entry
'Group=' line is a new entry
'PIDFile=' is edited to include the pid directory: /var/run/opchartsd/
'PermissionsStartOnly=' instructs that the service should only start with omkadmin as user and group this is necessary so the 'ExecStartPre=' lines are executed as root
'ExecStartPre=' lines are new entries
- opconfigd and opeventsd services have the same pattern of change, the difference being each services pid directory name
- omkd service has the same pattern of change, the difference again being for the pid directory name but has one extra line to edit for the path/to/pid parameter passed on the commandline to start omkd service:
- ExecStart=/usr/local/omk/script/opmantek.pl -f -p /var/run/omkd/opmantek.exe.pid -r
- There may be deeper issues where OMK services do need root privileges, for example to execute a command that will only run as root, but on the surface this has been working on ESXi instance pollux on vm named omkadmin@omk-vm9-centos7-runs-as-omkadmin-20220118 at IP 192.168.88.119 since 18 January 2022 ...
Logrotate
/etc/logrotate.d/omk-rotate.conf needs to be edited and each section of the logrotate script needs to include:
Code Block |
---|
create 0660 omkadmin omkadmin
su omkadmin omkadmin |
Check for issues using a cronjob
We allow group nmis as user omkadmin is a member of group nmis. If we exclude group nmis and only check for NOT group omkadmin, the following are returned soon after:
Code Block |
---|
sudo find /data/omk/ ! -group omkadmin ! -regex '/data/omk/var/lib/common/par-.+' -exec ls -lAh '{}' \;;
-rwxrwx---. 1 nmis nmis 21K Feb 5 23:15 /data/omk/var/opcore/registry/opConfig.json
-rwxrwx---. 1 nmis nmis 367 Feb 5 23:15 /data/omk/var/opcore/registry/open-audit.json
-rwxrwx---. 1 nmis nmis 105K Feb 5 23:15 /data/omk/var/opcore/registry/opEvents.json
-rwxrwx---. 1 nmis nmis 1.4K Feb 5 23:15 /data/omk/var/opcore/registry/opHA.json
-rwxrwx---. 1 nmis nmis 36K Feb 5 23:15 /data/omk/var/opcore/registry/opCharts.json
-rwxrwx---. 1 nmis nmis 112 Feb 5 23:14 /data/omk/var/opevents/file_state/_usr_local_nmis9_logs_event.log.json
-rwxrwx---. 1 nmis nmis 104 Feb 5 12:02 /data/omk/var/opevents/file_state/_usr_local_nmis9_logs_cisco.log.json |
Create cronjob /etc/cron.d/omk_check_omkadmin_user_group:
Code Block |
---|
PATH=/usr/local/sbin:/usr/local/bin:/sbin:/bin:/usr/sbin:/usr/bin
# m h dom mon dow user command
30 * * * * root find /data/omk ! -group omkadmin ! -group nmis ! -regex '/data/omk/var/lib/common/par-.+' -exec ls -lAh '{}' \;;
32 * * * * omkadmin find /data/omk ! -writable -exec ls -lAh '{}' \;;
34 * * * * root find /data/omk -perm /+2000 -exec ls -lAh '{}' \;;
36 * * * * root find /data/omk -perm /+4000 -exec ls -lAh '{}' \;;
40 * * * * root find /usr/local/omk ! -group omkadmin ! -group nmis ! -regex '/usr/local/omk/var/lib/common/par-.+' -exec ls -lAh '{}' \;;
42 * * * * omkadmin find /usr/local/omk ! -writable-exec ls -lAh '{}' \;;
44 * * * * root find /usr/local/omk -perm /+2000 -exec ls -lAh '{}' \;;
46 * * * * root find /usr/local/omk -perm /+4000 -exec ls -lAh '{}' \;; |
Reboot the VM to start clean - starts the OMK services that were stopped earlier
Code Block |
---|
sudo reboot; |
Weaning this VM off of SetGID (if it was ever set - originally set in first of these tests)
SetGID may be seen as a security risk so we don't SetGID bit anymore as part of the solution. Undo SetGID on /data/omk/ and /usr/local/omk directories if it was ever set:
Code Block |
---|
# stop services while we make this change:
sudo /usr/local/omk/bin/checkomkdaemons.sh stop;
# Unset SetGID on directories:
# /data/omk directory exists on the OMK NMIS9 VM:
sudo sh -c 'for F in $(find /data/omk -perm /+2000);do echo "${F}";chmod g-s "${F}";done;'
# Run this command a second time to confirm all have been fixed:
sudo sh -c 'for F in $(find /data/omk -perm /+2000);do echo "${F}";chmod g-s "${F}";done;'
# These command are exactly as for /data/omk:
sudo sh -c 'for F in $(find /usr/local/omk -perm /+2000);do echo "${F}";chmod g-s "${F}";done;'
# Run this command a second time to confirm all have been fixed:
sudo sh -c 'for F in $(find /usr/local/omk -perm /+2000);do echo "${F}";chmod g-s "${F}";done;'
# The following commands should be executed after any of the the above commands
# to ensure PAR directory structure is re-created with PAR's own permissions set:
# Set sticky bit on $PAR_GLOBAL_TMPDIR directory and only executable by root.
# This is a more secure implementation of the linux /tmp/ directory implementation which also uses sticky bit, but with chmod 1777:sudo chmod 1700 "${PAR_GLOBAL_TMPDIR:-FAIL_HERE}"
# Delete existing PAR subdirectories as we may have set incorrect permissions on this directory structure when executing the previous commands.
# The PAR subdirectories are re-created automatically by PAR upon being deleted (at execution of any PAR script exe by that user):
sudo rm -rf "${PAR_GLOBAL_TMPDIR:-FAIL_HERE}"/*
# reboot the VM to start clean - starts the OMK services that were stopped earlier:
sudo reboot; |