Troubleshooting problems with Infrastructure Management high availability

Consult this topic for troubleshooting information related to a high-availability deployment of Infrastructure Management.

• • Frequent fail over and fail back
  • To reconfigure the secondary server
  • To enable TLS mode
  • Related topics

Frequent fail over and fail back

In an application high availability deployment of TrueSight Infrastructure Management, there could be frequent fail over and fail back behavior based on cell being down either on primary or secondary.

The reason for frequent failover and failback due to cell down on primary or secondary might be because cell database (mcdb) is out of sync.

Resolution

Ensure that the secondary server is down and perform the following steps:

1. Delete all content in the installedDirectory\TrueSight\pw\server\var\<secondary_cell_name> folder of the secondary server.
2. Copy the installedDirectory\TrueSight\pw\server\var\<primary_cell_name>\mcdb file of the primary server to the above location on the secondary server.
3. Restart the secondary server.
4. Ensure that the primary server is running in active mode and secondary server in standby mode.

To reconfigure the secondary server

If you need to deploy and use a different secondary server instead of the original one, perform the following steps:

1. Install Infrastructure Management on the new secondary server.
2. Stop the primary server and the new secondary server.
3. On the primary server, edit the following files and replace the original secondary server host name with the new secondary server host name:
   • installedDirectory\pw\pronto\tmp\ha-generated.conf
   • installedDirectory\pw\server\etc\mcell.dir
   • installedDirectory\pw\pronto\data\admin\admin.dir
   • installedDirectory\pw\pronto\conf\federatedcacheserver-transport-tcp.xml
   • installedDirectory\pw\pronto\conf\clustermanager.xml
   • installedDirectory\pw\pronto\conf\cell_info.list
   • installedDirectory\pw\integrations\ibrsd\conf\IBRSD.dir
   • installedDirectory\pw\custom\conf\ha.conf
4. Copy the updated installedDirectory/TrueSight/pw/pronto/tmp/ha-generated.conf file from the primary server to the new secondary server.
5. On the new secondary server, run the following command 
   pw ha enable standby file=<location of ha-generated.conf file>
6. Copy the following folder from the primary server to the new secondary server:
    installedDirectory/TrueSight/pw/server/var/pncell_<hostname>#1
7. Rename the copied folder on the new secondary server to pncell_<hostname>#2
8. Copy the installedDirectory/TrueSight/pw/pronto/tmp/addremoteagent file from the primary server to the new secondary server.
9. Start the primary server.
10. On the primary server, run the addremoteagent file.
11. Start the secondary server, after the primary server is up and running.

To enable TLS mode

The switchToTLS.pl script does not work as expected when Infrastructure Management is configured for application high-availability. To resolve this issue, perform the following steps:

1. Stop the Infrastructure Management server.
2. Create a backup of the activemq-rar.rar file from the location installedDirectory\TrueSight\pw\wildfly\standalone\deployments.
3. Copy the ssl.activemq-rar.rar file from installedDirectory\TrueSight\pw\wildfly\store to the following locations and rename the file to activemq-rar.rar:
   
   • installedDirectory\TrueSight\pw\wildfly\standalone\deployments
   • installedDirectory\pw\tmp\Active
   • installedDirectory\pw\tmp\StandBy
4. Extract the amq-broker-config.xml file in each folder and ensure that the following content exists. If not, update the file:
   <amq:transportConnectors>
   <amq:transportConnector name="bppm-jms-broker-tcp-conn" uri="ssl://0.0.0.0:8093?
     transport.enabledProtocols=TLSv1.2&amp;transport.enabledCipherSuites=TLS_DHE_RSA_WITH_AES_128_GCM_SHA256,TLS_DHE_RSA_WITH_AES_128_CBC_SHA&amp;allowLinkStealing=true" allowLinkStealing="true" />
   </amq:transportConnectors>
5. Save the amq-broker-config.xml file if you made any changes to it and add it to the activemq-rar.rar file.
6. Restart the Infrastructure Management server.

Related topics

Infrastructure-Management-Server-high-availability-architectureInstalling and configuring the Infrastructure Management server for application high-availability