NMIS - Optimize nmis8/var For Speed - RAM Disk
Overview
Some customers may benefit by moving files that NMIS reads/writes to very often to a RAM Disk. We have seen this improve performance on pollers that have shared storage and 2,000+ nodes provisioned. The concept is that NMIS will be able to read/write to RAM faster than the shared storage. Most of the files that NMIS touches very often are found in the nmis8/var directory.
Any files written to RAM disk will not survive a reboot. After the server reboots the RAM disk will be empty.
Please review this Wikipedia article that addresses tmpfs:
This is an advanced configuration technique for large scale deployments with experienced sysadmin. The benefits for small scale deployments would be negligible.
Procedure
Step #1 - Determine Space Required
There are two questions that quickly come to mind:
- How big should the RAM disk be?
- How much RAM is available?
Check how much space nmis8/var is currently using; minus the nmis8/var/events directory. We will keep the nmis8/var/events directory on a hard drive because it contains stateful event history; we don't want to loose these if the server is rebooted.
[root@TEST var]# pwd /usr/local/nmis8/var [root@TEST var]# du -hs 2.6G . [root@TEST events]# pwd /usr/local/nmis8/var/events [root@TEST events]# du -hs 1.7G .
In this example we notice that the nmis8/var directory minus the nmis8/var/events directory is approximately 900MB. It appears that a RAM disk of 1GB would be sufficient.
How much RAM is available?
[root@TEST events]# free total used free shared buffers cached Mem: 37016616 25990916 11025700 1796 861860 15379752 -/+ buffers/cache: 9749304 27267312 Swap: 4169724 558624 3611100
In the example above there appears to be 11GB of RAM available.
The NMIS Self Test settings may need to be adjusted if the nmis8/var directory is running at a high rate of usage. Look for the following attributes in nmis8/conf/Config.nmis:
- 'selftest_min_diskfree_percent' => 10
- 'selftest_min_diskfree_mb' => 25
Step #2 - Create the RAM disk
The RAM disk mount point will need to be created. In this example we are using /media/ramdisk; be sure to mind the permissions.
The following commands will create the RAM disk, keep events on the hard drive, and ensure permissions are not lost. These commands should also need to be added to a script that runs at boot time. On Centos 6 /etc/rc.d/rc.local works well.
mount -t tmpfs -o size=1000M tmpfs /media/ramdisk/ mkdir -p /media/ramdisk/nmis8/var ln -s /data/nmis8/var/events /media/ramdisk/nmis8/var /usr/local/nmis8/admin/fixperms.pl
When creating the RAM disk the size parameter may be a number expressed with the suffix k, m or g (kilo, mega, and giga). It may also be expressed with the % sign. For example if you'd like to allow the ramdisk to consume up to 25 percent of the available RAM the following technique could be used.
mount -t tmpfs -o size=25% tmpfs /media/ramdisk
Keep in mind that the ramdisk will not immediately consume the amount of RAM specified. This is the most it will be allowed to consume should it need to. Also keep in mind that if the RAM is oversubscribed the ramdisk could be pushed into swap. If the ramdisk find's itself in swap space it defeats the purpose of this expedition, so do try to avoid this situation.
If you find yourself in a situation where you need to resize the ramdisk on the fly the following method may be useful.
# mount -o remount,size=4096M /media/ramdisk
This does not require a reboot and will preserve the files that reside on the ramdisk.
Use the df command to confirm that the ramdisk was created.
[root@TEST]# df -h Filesystem Size Used Avail Use% Mounted on /dev/mapper/vg_nmis64-lv_root 16G 11G 4.1G 73% / tmpfs 5.9G 0 5.9G 0% /dev/shm /dev/sda1 477M 75M 377M 17% /boot /dev/mapper/vg_nmis64_data-lv_data 246G 95G 139G 41% /data /dev/mapper/vg_nmis64-lv_var 20G 10G 8.3G 55% /var tmpfs 1000M 30M 971M 3% /media/ramdisk
Step #3 - Disable NMIS
We need to disable NMIS so that it will not attempt to modify the files as we point nmis8/var to the new RAM disk. This is easily accomplished by placing a file called 'NMIS_IS_LOCKED' in the nmis8/conf directly as in the following example.
[root@TEST]# touch /usr/local/nmis8/conf/NMIS_IS_LOCKED
Once the lock file is in place check the server for any nmis.pl processes.
[root@TEST]# ps -ef | grep nmis.pl
When none are found continue
Step #4 - Point nmis8/var at the RAM Disk
Since we are planning to put a sym link in /usr/local/nmis8/ for /var pointing at the new RAM disk; verify that in /usr/local/nmis8/conf/Config.nmis nmis8/var points to /usr/local/nmis8/var. Here's a sample of the directories configuration in that file.
'directories' => { '<menu_base>' => '<nmis_base>/menu', '<nmis_admin>' => '<nmis_base>/admin', '<nmis_backups>' => '<nmis_data>/backups', '<nmis_base>' => '/usr/local/nmis8', '<nmis_bin>' => '<nmis_base>/bin', '<nmis_cgi>' => '<nmis_base>/cgi-bin', '<nmis_conf>' => '<nmis_base>/conf', '<nmis_data>' => '<nmis_base>', '<nmis_logs>' => '<nmis_base>/logs', '<nmis_menu>' => '<nmis_base>/menu', '<nmis_models>' => '<nmis_base>/models', '<nmis_var>' => '<nmis_data>/var', 'config_logs' => '<nmis_logs>/config', 'database_root' => '<nmis_data>/database', 'json_logs' => '<nmis_logs>/json', 'log_root' => '<nmis_logs>', 'mib_root' => '<nmis_base>/mibs', 'plugin_root' => '<nmis_conf>/plugins', 'report_root' => '<nmis_base>/htdocs/reports', 'script_root' => '<nmis_conf>/scripts', 'web_root' => '<nmis_base>/htdocs' },
Create the sym link in /usr/local/nmis8/ sending /var to the RAM disk.
ln -s /media/ramdisk/nmis8/var/ /usr/local/nmis8
Step #5 - Re-enable NMIS
Remove the previously created NMIS lock file.
[root@TEST]# rm /usr/local/nmis8/conf/NMIS_IS_LOCKED
Step #6 - Verify
move to the /usr/local/nmis8/var directory and execute a 'df -h .' command. This command should reveal that we are indeed on the ramdisk.
[root@TEST ~]# cd /usr/local/nmis8/var [root@TEST var]# df -h . Filesystem Size Used Avail Use% Mounted on tmpfs 1000M 30M 971M 3% /media/ramdisk [root@TEST var]#
- Take a look in the /usr/local/nmis8/var directory and confirm that node.json and view.json files have been created.
- Verify that the NMIS GUI is functioning normally.
- Review the nmis8/logs/nmis.log file
- Check the root user mailbox