Automatically copy opReports to another server using SFTP
Table of contents
Requirement
A report was required for billing when the usage exceeded the contract, sometimes referred to as 95th percentile billing, this report will be sent to another department for use in their reporting. This report needed to have specific columns and headings for use in that department's systems. A new more general report was created in opReports and this report needs to be transformed and transferred using SFTP.
Solution
Overview
Two new reports were added to opReports 4.4.1 for Burst Billing Report or Bandwidth Exceeds Contract Report, currently when opReports generates these reports, it will look for the existence of a script called /usr/local/omk/bin/opreports-sftp.pl, if it finds this script file it will execute that script file to operate on the reports generated by opReports, transform them and then transfer the files as required.
Installation
The code for this solution is included in the NMIS9 contrib folder which is available in the installation or from NMIS9@GitHub, it will be in the folder nmis9/contrib/reports
This particular script will need to be linked into the /usr/local/omk/bin folder as that is where opReports will look for it.
Some handy commands (sudo might be required):
sudo mkdir /usr/local/nmis9/util sudo ln -s /usr/local/nmis9/contrib/reports/opreports-sftp.pl /usr/local/omk/bin sudo cp /usr/local/nmis9/contrib/reports/FtpExport.nmis /usr/local/nmis9/conf sudo /usr/local/nmis9/bin/nmis-cli act=fixperms
Update Configuration File
You will need to edit /usr/local/nmis9/conf/FtpExport.nmis and update your settings, the file looks like this:
%hash = ( "ftp_server" => "your.ftp.server.domain.com", "ftp_user" => "ftp_user", "ftp_password" => "dapasswordsecretthing", "ftp_directory" => "/tmp", "ftp_log_directory" => "/tmp" );
The configuration settings are:
Setting | Description |
---|---|
ftp_server | The IP address or FQDN of the target server. |
ftp_user | The FTP username to use. |
ftp_password | The FTP password to use. |
ftp_directory | Where to put the files. |
ftp_log_directory | Where to put the FTP log files. |
Future Plans
The team plan to make a more general version of this feature so that when a report is generated by opReports, it can call a script at completion which can then perform any required action with those files, this could be transformations, file transfers, automated loading into a data lake, emails, etc.