As part of synchronizing the nondatabase opennms files from VIP to non-VIP, we cron a utility to run every midnight. The following cron does this job:
0 0 * * * /opt/opennms/contrib/failover/scripts/sync.sh >> /opt/opennms/logs/failover.log 2>&1
This script uses the csync2 utility internally to synchronize the file between the servers.
At times, a user might notice certain files missing in the non-VIP node even though the cron job has run. The user can also see similar errors in /var/log/csync2.log on the VIP node.
/var/log/csync2.log
ERROR: Connection to remote host failed. ERROR: Connection to remote host failed. While syncing file /var/opennms/rrd/snmp/10/bme0_32768-000bcafe0000/mib2-X-interfaces-pkts.jrb: ERROR from peer space-0319062015000027: Connection closed. ERROR from peer space-0319062015000027: Connection closed. ERROR: Connection to remote host failed. ERROR: Connection to remote host failed. ERROR: Connection to remote host failed. ERROR: Connection to remote host failed. Finished with 8 errors.
The csync2 utility uses TCP port 30865. The user needs to open TCP port 30865 on both hosts (VIP and non-VIP) so that the other host can connect.
To determine whether a node is a VIP node, execute ifconfig eth0:0 on the CLI to see if it returns an IP address. If it does, it is a VIP node.
ifconfig eth0:0
If a firewall rule to open the TCP port is already in place, check the status of csync2 service on the non-VIP node. This can be verified with the following command:
[root@space-005056adfcb6 ~]# chkconfig --list csync2 csync2 off
The csync2 service should be turned on for the synchronization to work. If it is off (as shown in the output above), the connection to the remote host will not go through. The same error messages can be seen in /var/log/csync2.log of the VIP node.
To enable the service, do the following:
[root@space-005056adfcb6 ~]# chkconfig csync2 on [root@space-005056adfcb6 ~]# service xinetd restart Stopping xinetd: [ OK ] Starting xinetd: [ OK ] [root@space-005056adfcb6 ~]# chkconfig --list csync2 csync2 on
Then manually run the script /opt/opennms/contrib/failover/scripts/sync.sh and check if the files are getting synchronized.
/opt/opennms/contrib/failover/scripts/sync.sh