Automatically copy opReports to another server using SFTP

Table of contents

Requirement

A report was required for billing when the usage exceeded the contract, sometimes referred to as 95th percentile billing, this report will be sent to another department for use in their reporting.  This report needed to have specific columns and headings for use in that department's systems.  A new more general report was created in opReports and this report needs to be transformed and transferred using SFTP.

Solution

Overview

Two new reports were added to opReports 4.4.1 for Burst Billing Report or Bandwidth Exceeds Contract Report, currently when opReports generates these reports, it will look for the existence of a script called /usr/local/omk/bin/opreports-sftp.pl, if it finds this script file it will execute that script file to operate on the reports generated by opReports, transform them and then transfer the files as required.

Installation

The code for this solution is included in the NMIS9 contrib folder which is available in the installation or from NMIS9@GitHub, it will be in the folder nmis9/contrib/reports

This particular script will need to be linked into the /usr/local/omk/bin folder as that is where opReports will look for it.

Some handy commands (sudo might be required):

sudo mkdir /usr/local/nmis9/util
sudo ln -s /usr/local/nmis9/contrib/reports/opreports-sftp.pl /usr/local/omk/bin
sudo cp /usr/local/nmis9/contrib/reports/FtpExport.nmis /usr/local/nmis9/conf
sudo /usr/local/nmis9/bin/nmis-cli act=fixperms

Update Configuration File

You will need to edit /usr/local/nmis9/conf/FtpExport.nmis and update your settings, the file looks like this:

%hash = (
  "ftp_server" => "your.ftp.server.domain.com",
  "ftp_user" => "ftp_user",
  "ftp_password" => "dapasswordsecretthing",
  "ftp_directory" => "/tmp",
  "ftp_log_directory" => "/tmp"
);

The configuration settings are:

SettingDescription
ftp_serverThe IP address or FQDN of the target server.
ftp_userThe FTP username to use.
ftp_passwordThe FTP password to use.
ftp_directoryWhere to put the files.
ftp_log_directoryWhere to put the FTP log files.

Future Plans

The team plan to make a more general version of this feature so that when a report is generated by opReports, it can call a script at completion which can then perform any required action with those files, this could be transformations, file transfers, automated loading into a data lake, emails, etc.