Replication Setup and Configuration Guide

...

Table of Contents

minLevel	1
maxLevel	6
include
outline	true
indent
style	none
exclude
type	list
class
printable	true

...

Term	Meaning
Main Primary	server instance running opHA with role = 'primary_master'
Secondary Primary	server instance running opHA with role = 'master'. Also referred to as just 'primary'.
Peer	server instance running opHA with role = 'poller' or 'mirror'. It can optionally be of type 'streaming' which means that it is linked to an ophad instance.
Poller	server instance running opHA with role = 'poller'
Mirror	server instance running opHA with role = 'mirror'. It is paired with a poller and polls the same devices as that poller, however its data is not sync'd with primary unless the poller is offline.
Producer	an instance of the ophad process running in 'producer' mode (as per configuration). This will typically be running on the same server instances as the primary.
Consumer	an instance of the ophad process running in 'consumer' mode (as per configuration). This will typically be running on the same server instance as the poller (or mirror)

Software Installation

opHA-MB 5.1.0 can be installed on infrastructure

New to opHA/opHA-MB ----- Install to new servers
Running opHA 4.X ----- Upgrade existing servers, running opHA 4.x

Installation of opHA-5.1.0 that does not have opHA software installed

Prerequisites

IMPORTANT: All the servers should be setup with the following

...

d. opHA-MB licenses to be installed.

Editing the Role of the instance in opCommon.json

The default value of opha_role in opCommon.json is Standalone

...

“Main Primary”
“Primary”
“Poller”
“Mirror”

Installing opHA-MB license

Install the opHA-MB for NMIS & opCharts license on the Main Primary server only.

...

You need to enter a License Key in the Modules → opLicensing as below on the Main Primary.

...

Download Software

Download the required software onto each server.

Install Software

We will be using the below example host names throughout this guide.

...

During the installation of the arbiter server you must answer 'n' at the following prompt

...

NATS Setup

This configuration should be applied to the Main Primary, Arbiter and Secondary Primary servers ONLY.

Edit Configuration: Update the /etc/nats-server.conf file with the following settings:
1. On Main Primary:
  1. server_name: opha-dev1.opmantek.net
  2. host: "opha-dev1.opmantek.net"
  3. routes: [
    # secondary primary
    "nats://opha-dev6.opmantek.net:6222"
    # arbiter
    "nats://opha-dev2.opmantek.net:6222"
    ]
2. On Secondary Primary:
  1. server_name: opha-dev6.opmantek.net
  2. host: "opha-dev6.opmantek.net"
  3. routes: [
    # main primary
    "nats://opha-dev1.opmantek.net:6222"
    # arbiter
    "nats://opha-dev2.opmantek.net:6222"
    ]
3. On Arbiter:
  1. server_name: opha-dev2.opmantek.net
  2. host: "opha-dev2.opmantek.net"
  3. routes: [
    # main primary
    "nats://opha-dev1.opmantek.net:6222"
    # secondary primary
    "nats://opha-dev6.opmantek.net:6222"
    ]

Sample file for Main Primary:-

Code Block

language	json

server_name: "opha-dev1.opmantek.net"  #The local server  
http_port: 8222 
listen: 4222 
jetstream: enabled 

#tls {
#    cert_file: "<path>"
#    key_file:  "<path>"
#    #ca_file:   "<path>"
#    verify: true
#}

log_file: "/var/log/nats-server.log"

accounts { 
  $SYS { 
    users: [ 
      { user: "admin",
        pass: "password" 
      } 
    ] 
  } 
  ophad: { 
    users: [ 
        { user: "omkadmin", password: "op42opha42" } 
    ] 
    jetstream: enabled 
  } 
} 

jetstream { 
  store_dir: "/opt/nats/storage" 
  max_memory_store: 1028M 
  max_file_store: 1028M 
} 
cluster { 
  name: "C1" 
  host: "opha-dev1.opmantek.net"  #The current host 
  # the current server 
  listen: "0.0.0.0:6222" 
  routes: [ 
    # secondary primary
    "nats://opha-dev6.opmantek.net:6222" 
    # server with the arbiter 
    "nats://opha-dev2.opmantek.net:6222" 
    # other servers 
  ] 
}

Common Configuration for All Servers

Update `/usr/local/omk/conf/opCommon.json`

Backup Configuration: Make a backup copy of the opCommon.json file before making any changes.
Add NATS Cluster Configuration: Ensure the following configuration is included in opCommon.json on all servers. In the below example we have added the Main Primary, arbiter and Secondary Primary FQDNs and port/s to be utilized in the nats_cluster attribute, you can find this in the database section.
Code Block
language json
"nats_cluster": [ "opha-dev1.opmantek.net", "opha-dev2.opmantek.net", "opha-dev6.opmantek.net" ]

Server-Specific Configuration

Update `/usr/local/omk/conf/opCommon.json` for Main Primary and Secondary Primary only. For key “database” add the following information:

db_mongo_cluster is cluster of ‘Main Primary’ and ‘Secondary Primary’

Code Block

language	json

"database" : {
    "db_replica_set": "rs1", 
    "db_mongo_cluster": [ "opha-dev1.opmantek.net", "opha-dev6.opmantek.net" ], 
    "nats_num_replicas": 3, 
    "nats_cluster": [ "opha-dev1.opmantek.net", "opha-dev2.opmantek.net", "opha-dev6.opmantek.net" ]
  }

MongoDB Setup (Main Primary Only)

The following steps should be performed only on the Main Primary MongoDB instance.

Initialize MongoDB Replica Set

Connect to MongoDB using the mongosh command and configure the replica set:

...

Code Block
rs.initiate({ _id: "rs1", version: 1, members: [ { _id: 0, host : "opha-dev1.opmantek.net:27017", priority: 2 }, { _id: 1, host : "opha-dev6.opmantek.net:27017", priority: 1 } ] })

Set Default Write Concern

Run the following command to set the default write concern:

Code Block
db.adminCommand({ "setDefaultRWConcern": 1, "defaultWriteConcern": { "w": 1 } })

Add Arbiter

Run the following command to add an arbiter to the replica set:

Code Block
rs.addArb("opha-dev2.opmantek.net:27018")

Restart mongod on the Main Primary, Second Primary and Arbiter

Run the following command on the Main Primary, Secondary Primary, and Arbiter servers ONLY:

Code Block
sudo systemctl restart mongod

Starting Services

Start NATS

Start the NATS service on the Main Primary, Secondary Primary, and Arbiter servers ONLY:

Code Block
systemctl start nats-server

Restart mongod on Pollers and Mirrors only

After configuration changes, restart mongod on all Mirrors and Pollers servers ONLY:

Code Block
systemctl restart mongod

Restart services on all servers

After configuration changes have been made, you will need to restart the relevant FirstWave module daemons applicable to your server.

...

Code Block
systemctl restart nmis9d opchartsd opeventsd omkd ophad

Start the Poller and Mirrors

Execute the following command on the poller and mirror servers.

Code Block
cd /usr/local/omk/bin ./ophad cmd producer start

Final Steps

Accept EULAs: Make sure when you login to each server to confirm that all End User License Agreements (EULAs) are accepted.

Discover the Peers

Click on Peers in opHA-MB portal on the Main Primary (http://<fqdn of Main Primary>/en/omk/opHA/peers)

...

Proceed to next step after discovering all the Peers.

...

Upgrade to opHA-5.1.0 from opHA 4.X

Prerequisites

IMPORTANT: All the servers should be setup with the following

...

d. opHA-MB licenses to be installed.

Editing the Role of the instance in opCommon.json

The default value of opha_role in opCommon.json is Standalone

...

“Main Primary”
“Primary”
“Poller”
“Mirror”

Installing opHA-MB license

Install the opHA-MB for NMIS & opCharts license on the Main Primary server only.

...

You need to enter a License Key in the Modules → opLicensing as below on the Main Primary.

...

Download Software

Download the required software onto each server.

Install Software

IMPORTANT: All the servers should be setup with the following

...

During the installation of the arbiter server you must answer 'n' at the following prompt

...

NATS Setup

This configuration should be applied to the Main Primary, Arbiter and Secondary Primary servers ONLY.

Edit Configuration: Update the /etc/nats-server.conf file with the following settings:

Code Block

language	json

server_name: "opha-dev1.opmantek.net"  #The local server  
http_port: 8222 
listen: 4222 
jetstream: enabled 

#tls {
#    cert_file: "<path>"
#    key_file:  "<path>"
#    #ca_file:   "<path>"
#    verify: true
#}

log_file: "/var/log/nats-server.log"

accounts { 
  $SYS { 
    users: [ 
      { user: "admin",
        pass: "password" 
      } 
    ] 
  } 
  ophad: { 
    users: [ 
        { user: "omkadmin", password: "op42opha42" } 
    ] 
    jetstream: enabled 
  } 
} 

jetstream { 
  store_dir: "/opt/nats/storage" 
  max_memory_store: 1028M 
  max_file_store: 1028M 
} 
cluster { 
  name: "C1" 
  host: "opha-dev1.opmantek.net"  #The current host 
  # the current server 
  listen: "0.0.0.0:6222" 
  routes: [ 
    # secondary primary
    "nats://opha-dev6.opmantek.net:6222" 
    # server with the arbiter 
    "nats://opha-dev2.opmantek.net:6222" 
    # other servers 
  ] 
}

Common Configuration for All Servers

Update `/usr/local/omk/conf/opCommon.json`

Backup Configuration: Make a backup copy of the opCommon.json file before making any changes.
Add NATS Cluster Configuration: Ensure the following configuration is included in opCommon.json on all servers. In the below example we have added the Main Primary, arbiter and Secondary Primary FQDNs and port/s to be utilized in the nats_cluster attribute, you can find this in the database section.
Code Block
language json
"nats_cluster": [ "opha-dev1.opmantek.net", "opha-dev2.opmantek.net", "opha-dev6.opmantek.net" ]

Server-Specific Configuration

Update `/usr/local/omk/conf/opCommon.json` for Main Primary and Secondary Primary only

Code Block

language	json

"db_replica_set": "rs1", 
"db_mongo_cluster": [ "opha-dev1.opmantek.net", "opha-dev6.opmantek.net" ], 
"nats_num_replicas": 3, 
"nats_cluster": [ "opha-dev1.opmantek.net", "opha-dev2.opmantek.net", "opha-dev6.opmantek.net" ]

MongoDB Setup (Main Primary Only)

The following steps should be performed only on the Main Primary MongoDB instance.

Initialize MongoDB Replica Set

Connect to MongoDB using the mongosh command and configure the replica set:

...

Code Block
rs.initiate({ _id: "rs1", version: 1, members: [ { _id: 0, host : "opha-dev1.opmantek.net:27017", priority: 2 }, { _id: 1, host : "opha-dev6.opmantek.net:27017", priority: 1 } ] })

Set Default Write Concern

Run the following command to set the default write concern:

Code Block
db.adminCommand({ "setDefaultRWConcern": 1, "defaultWriteConcern": { "w": 1 } })

Add Arbiter

Run the following command to add an arbiter to the replica set:

Code Block
rs.addArb("opha-dev2.opmantek.net:27018")

Restart mongod on the Main Primary, Second Primary and Arbiter

Run the following command on the Main Primary, Secondary Primary, and Arbiter servers ONLY:

Code Block
sudo systemctl restart mongod

Starting Services

Start NATS

Start the NATS service on the Main Primary, Secondary Primary, and Arbiter servers ONLY:

Code Block
systemctl start nats-server

Restart mongod on Pollers and Mirrors only

After configuration changes, restart mongod on all Mirrors and Pollers servers ONLY:

Code Block
systemctl restart mongod

Restart services on all servers

After configuration changes have been made, you will need to restart the relevant FirstWave module daemons applicable to your server.

...

Code Block
systemctl restart nmis9d opchartsd opeventsd omkd

Discover the Peers

Click on Peers in opHA-MB portal on the Main Primary (http://<fqdn of Main Primary>/en/omk/opHA/peers)

...

Proceed to next step after discovering all the Peers.

Restart opHA service on the Main Primary

Code Block
sudo systemctl restart ophad

Start the Poller and Mirrors

Execute the following command on the poller and mirror servers.

Code Block
cd /usr/local/omk/bin ./ophad cmd producer start

Final Steps

Accept EULAs: Make sure when you login to each server to confirm that all End User License Agreements (EULAs) are accepted.

Version	Old Version 12	New Version 13
Changes made by	Amith Bhonsle	Shankar Nagarajan
Saved on	Jun 27, 2025	Jul 27, 2025

Content Comparison

Versions Compared

Key

Replication Setup and Configuration Guide

Software Installation

Installation of opHA-5.1.0 that does not have opHA software installed

Prerequisites

Editing the Role of the instance in opCommon.json

Installing opHA-MB license

Download Software

Install Software

NATS Setup

Common Configuration for All Servers

Update /usr/local/omk/conf/opCommon.json

Server-Specific Configuration

Update /usr/local/omk/conf/opCommon.json for Main Primary and Secondary Primary only. For key “database” add the following information:

MongoDB Setup (Main Primary Only)

Initialize MongoDB Replica Set

Set Default Write Concern

Add Arbiter

Restart mongod on the Main Primary, Second Primary and Arbiter

Starting Services

Start NATS

Restart mongod on Pollers and Mirrors only

Restart services on all servers

Start the Poller and Mirrors

Final Steps

Discover the Peers

Upgrade to opHA-5.1.0 from opHA 4.X

Prerequisites

Editing the Role of the instance in opCommon.json

Installing opHA-MB license

Download Software

Install Software

NATS Setup

Common Configuration for All Servers

Update /usr/local/omk/conf/opCommon.json

Server-Specific Configuration

Update /usr/local/omk/conf/opCommon.json for Main Primary and Secondary Primary only

MongoDB Setup (Main Primary Only)

Initialize MongoDB Replica Set

Set Default Write Concern

Add Arbiter

Restart mongod on the Main Primary, Second Primary and Arbiter

Starting Services

Start NATS

Restart mongod on Pollers and Mirrors only

Restart services on all servers

Discover the Peers

Restart opHA service on the Main Primary

Start the Poller and Mirrors

Final Steps

Update `/usr/local/omk/conf/opCommon.json`

Update `/usr/local/omk/conf/opCommon.json` for Main Primary and Secondary Primary only. For key “database” add the following information:

Update `/usr/local/omk/conf/opCommon.json`

Update `/usr/local/omk/conf/opCommon.json` for Main Primary and Secondary Primary only