Operating System and network issues


The TrueSight Presentation Server console also known as TrueSight console is hosted on the Presentation Server.

This topic lists a few of the Operating System and network issues observed in TrueSight Presentation Server.



Check open ports on Windows server 

  • To display all open ports, open command prompt navigate to Start > Run > cmd, type netstat and press enter.
  • To list all listening ports, use the following command:

     netstat -an |find /i listening
  • To find specified open port, use find switch. 
    For example, to find if the port 1900 is open or not, use netstat -an |find /i “1900“

Check open ports in Linux server 

  • Open a Linux terminal application.
  • Use the ss command to display all open TCP and UDP ports in Linux.
  • Alternatively, use the netstat command to list all ports in Linux.
  • Apart from the ss and  netstat command, you can use the lsof command to list open files and ports on Linux based system.


Symptoms

tssh process start database command is not working.

Cause

User does not have the required permissions on directory pointed by the environment variable %TRUESIGHTPSERVER_HOME%.

Solution

  1. Open Windows Explorer in the TrueSight Presentation Server machine.
  2. Navigate to the directory pointed by the environment variable %TRUESIGHTPSERVER_HOME% .(i.e., C:\Program Files\BMC Software\TrueSightPServer)
  3. Right-click on the TrueSightPServer directory name to get the properties.
  4. Click on Security.
  5. From the Groups or user name section,click To change permissions.
  6. Click Add to add the admin user(either local or domain) into to the list of users.
  7. Click Apply and Ok to save the changes.
  8. Log out and log in again with the same user to the machine and run the command to start the PostgreSQL database.


When the secondary TrueSight Presentation Server node is newly added into High Availability (HA) mode to an existing TrueSight Presentation Server standalone environment- data such as policies, connected agents, components, etc. are missing in the TrueSight console post failover from primary TrueSight Presentation Server to Secondary TrueSight Presentation Server.

The copy snapshot from primary to secondary database is not successful.

  1. Stop both the nodes.
  2. Take a backup of the %TRUESIGHTPSERVER_HOME%\truesightpserver\data\pgsql folder on secondary and then delete it.
  3. Copy the %TRUESIGHTPSERVER_HOME%\truesightpserver\data\pgsql folder from primary node to secondary node.
  4. Start the secondary node.
  5. Log in to the secondary TrueSight Presentation Server.
    The missing data is now replicated from primary node to secondary node.


TrueSight Infrastructure Management Change Management integration is not registered in TrueSight Presentation Server.





Error:

The following error appears in TrueSight Presentation Server logs directory of session.stdout file:

Received valid Access Point URL: http://ARServername:8080/arsys/services/ARService?server=tsmdx05&webService=CAI_ChangeManagementEventsSubscription_WS
UDDI Lookup Result ITSM WS URL : http://ARServername:8080/arsys/services/ARService?server=tsmdx05&webService=CAI_ChangeManagementEventsSubscription_WS
Authentication object is created.
About to register the current instance. { emails= null instanceName=TrueSight Presentation Server:truhx01a:8043 authURL=https://truhx01a:8043/tsws/10.0/api/authenticate/login eventURL=https://TrueSight Presentation Serverservername:8043/tsws/10.0/api/CI/{ReconId}/event?idType=ReconId reason=Activated from TrueSight Presentation Server UI}
org.apache.catalina.core.StandardWrapperValve invoke
SEVERE: Servlet.service() for servlet [Resteasy] in context with path [/tsws] threw exception [org.jboss.resteasy.spi.UnhandledException: java.lang.NoSuchMethodError: org.apache.axiom.soap.SOAPFaultCode.getValueAsQName()Ljavax/xml/namespace/QName;] with root cause
java.lang.NoSuchMethodError: org.apache.axiom.soap.SOAPFaultCode.getValueAsQName()Ljavax/xml/namespace/QName;

Communications from TrueSight Infrastructure Management to TrueSight Presentation Server is blocked.

In this case, it is port 8043.

Allow the firewall port to communicate with TrueSight Presentation  Server.



TrueSight Presentation Server and TrueSight Infrastructure Management Server, both are installed in high availability.TrueSight Infrastructure Management is successfully registered to TrueSight Presentation Server when primary TrueSight Infrastructure Management node is active.

After failover, the TrueSight Infrastructure Management to secondary server is not showing on component page of TrueSight console.








Multiple NIC cards










  • Check if multiple NIC card is configured on TrueSight Infrastructure Management Server with different IP addresses.
    If yes, check on which IP, port 8093 is establishing the connection.

Windows

netstat -aon | findstr 8093

Linux

netstat -aon | grep 8093


For Example:

NIC1:- TSIM1.test.com - IPADDRESS_1
NIC2:- TSIM1.test1.com - IPADDRESS_11

NIC1:- TSIM2.test.com - IPADDRESS_2
NIC2:- TSIM2.test.2com - IPADDRESS_21


If 8093 is binding on IPADDRESS_1, then change the following  parameter:
TCP bind_addr="${jgroups.tcp.address:127.0.0.1}" to TCP bind_addr="IPADDRESS_1"

On TrueSight Infrastructure Management
in/pw/pronto/conf/federatedcacheserver-transport tcp.xml


On TrueSight Presentation Server

TCP_bind_addr updates in
%TRUESIGHTPSERVER_HOME%/truesightpserver/conf/ha/cache-replicator.conf and 
%TRUESIGHTPSERVER_HOME%/truesightpserver/conf/ha/leaderselect.conf.

  • Restart the TrueSight Infrastructure Management  and TrueSight Presentation Server.



A new install and/or upgrade of TrueSight Presentation Server to 11.3.04 version fails to create the PostgreSQL database on Windows 2012R2.










%COMSPEC% environment is not configured or not correctly configured.










  1. Check the %COMSPEC% environment variable to find an incorrect %COMSPEC% environment variable that prevents the execution of initdb.
  2. In the command prompt,run the following command:

    echo %COMSPEC%
  3. The result appears like below:
    C:\Windows\system32\cmd.exe
    (Where C:\Windows is your Windows install location).

    If the result doesn't appears like above, from the system control panel>environment section, check COMSPEC for the wrong value.

    1. Make a system restore point and
      back up your computer then
      change it in the system environment settings so that it points to the correct location of cmd.exe
    2. You can test it to make sure if it is correct.

      Open a new cmd.exe prompt and run the following command:

      %COMSPEC%" /C "echo test ok
    3. It prints test ok.

    The installer shows that the specified account is an administrator, but it is not the actual case.

    Most likely, the specified account is an administrator or power user, even though you aren't aware of it. The check used by the installer specifically checks for membership in the Administrators or Power users group.

    Check using Local users and Groups. Open the Administrators group to see who's a member. Then check any groups (domain or local) that are a member of the Administrators group, and any group members of that group etc.. PostgreSQL checks any level of nested groups.



After installing secondary node of a TrueSight Presentation Server in High Availability (HA) setup, secondary TrueSight Presentation Server services fail to start.

The /etc/hosts file on both servers have both the hostname and FQDN of the primary and secondary nodes configured.

This is causing the primary server PostgreSQL database to start with hostname instead of FQDN.

During the installation on the secondary node, the installer instructs the tssh ha copysnapshot command to get a copy of the primary node database.

If this command fails, then the data/pgsql folder shall be empty in the secondary node.

This happens because the /etc/hosts file on both servers are configured in the following manner:

<Primary_IP> <Primary_hostname> <Primary_FQDN>
<Secondary_IP> <Secondary_hostname> <Secondary_FQDN>

This causes the PostgreSQL database on the primary server to initialize only with the hostname rather than the FQDN. So when the secondary TrueSight Presentation Server tries to contact the primary database with the FQDN, it fails to connect.

Following steps fix the issue:

  1. Update the /etc/hosts file on both nodes to reflect in the below manner:

    <Primary_IP> <Primary_FQDN>
     <Secondary_IP> <Secondary_FQDN>

  2. Restart all services on primary node and wait for it to initialize.
  3. Start all services on secondary node. The copy snapshot command starts working, after which all the secondary TrueSight Presentation Server services starts up successfully.


The tssh ha copysnapshot command fails in the TrueSight Presentation Server.




The %TRUESIGHTPSERVER_HOME%/truesightpserver/logs/db-remote-copy.log contains the following error:

pg_basebackup: could not connect to server: FATAL: no pg_hba.conf entry for replication connection from host "<TrueSight Presentation Server_Node1/Node2_IP>", user "<TrueSight Presentation Server_Database_User>", SSL off



Verify the following if:

  • Telnet is working on port 5432.
  • Ping is working from primary TrueSight Presentation Server to secondary TrueSight Presentation Server .
  • Primary and secondary hostname of TrueSight Presentation Server are resolvable with the nslookup command.
  • Reverse lookup is working for both primary and secondary TrueSight Presentation Server.

If the above details are correct, do the following:

  1.  Stop secondary TrueSight Presentation Server node by running tssh server stop
  2. Stop primary TrueSight Presentation Server node by running 
    tssh server stop
  3.  Take backup of %TRUESIGHTPSERVER_HOME%\truesightpserver\data\pgsql\pg_hba.conf
  4.  Open %TRUESIGHTPSERVER_HOME%\truesightpserver\data\pgsql\pg_hba.conf and add the IP address of secondary and primary TrueSight Presentation Server as below:

    hostnossl replication <TrueSight Presentation Server_Database_User> <TrueSight Presentation Server_Primary_IP>/32 trust
    hostnossl replication <TrueSight Presentation Server_Database_User> <TrueSight Presentation Server_Secondary_IP>/32 trust

    For Example: hostnossl replication TrueSight Presentation Serveradmin 10.1.2.34/32 trust

  5. Start the primary server by running
    tssh server start
  6. Verify if primary server showing as active by running 
    tssh ha status
  7. Once primary is up, start the secondary server by running 
    tssh server start
  8. Verify if secondary is showing as standby by running 
    tssh ha status


Symptoms

The tssh ha status command  in the secondary server shows error.

The secondary server is running and primary server is also running and active.

Cause

Error: Server is not reachable/unavailable: Connect to localhost:443 [localhost/127.0.0.1, localhost/0:0:0:0:0:0:0:1] failed: Connection refused: connect

Permission issue on the server.xml and ha.conf files.

Solution

  • Check the %TRUESIGHTPSERVER_HOME%\truesightpserver\logs\tssh.log in the secondary server if you are getting the following error:

    ERROR .716 [main] c.b.t.a.u.FileManagerUtil BMC_TS-CA000020E Unable to read HTTP port info from file [D:\Program Files\BMC Software\TrueSightPServer\truesightpserver\modules\tomcat\conf\server.xml] Exception while reading Tomcat's configuration file.
    java.io.FileNotFoundException: D:\Program Files\BMC Software\TrueSightPServer\truesightpserver\modules\tomcat\conf\server.xml (Access is denied)

    INFO .934 [main] c.b.t.p.c.p.CPAppPropertiesInitializer Completed Properties Cache initialization
    ERROR .640 [main] c.b.t.a.p.AppProperties BMC_TS-CA000027E Exception while checking user details
    java.io.FileNotFoundException: D:\Program Files\BMC Software\TrueSightPServer\truesightpserver\conf\ha\ha.conf (Access is denied)

  • If yes, ensure that you have granted the read/execute permission to the server.xml and ha.conf files.




Integration Service (IS) clusters created in primary TrueSight Presentation Server gets erased after failover to secondary TrueSight Presentation Server.

In a TrueSight Presentation Server High-Availability (HA) setup, IS clusters created in the primary console gets erased after failover to secondary. A different symptom is observed - post failover from primary to secondary, all the old entries that are previously created for the IS clusters, shown together in the IS cluster section.


Two Network Interface Cards (NIC) configured on the server causing the jgroup to use the wrong IP address for communication and this causes the cache replication to fail during the failover to secondary.



The cache-replicator.conf file needs to be edited by replacing the Fully Qualified Domain Name (FQDN) of the server with the default IP address of the server on both the primary and secondary nodes.

This will ensure that the jgroup communicates via the correct IP address only.

Following are the steps to fix the issue:

  1. Stop the services on both the nodes.
  2. Make the following modifications:
Primary TrueSight Presentation Server


  1. Back up of the file %TRUESIGHTPSERVER_HOME%/truesightpserver/conf/ha/cache-replicator.conf and place the backup file in a different location.
  2. Open the file %TRUESIGHTPSERVER_HOME%/truesightpserver/conf/ha/cache-replicator.conf in edit mode.
  3. Find the line that starts with <TCP bind_addr="${jgroups.tcp.address:<Primary__FQDN>}"
  4. Replace <Primary__FQDN> with <Primary__Default_IP>.
     After the changes the line appears as :
    <TCP bind_addr="${jgroups.tcp.address:<Primary__Default_IP>}"
  5. Find line that starts with <TCPPING ergonomics="false" initial_hosts="<Primary__FQDN>[7800],<Secondary__FQDN>[7800]"
  6. Replace <Primary__FQDN> with <Primary__Default_IP> and <Secondary__FQDN> with <Secondary__Default_IP>.
    After the changes the line appears as:
    TCPPING ergonomics="false" initial_hosts="<Primary__Default_IP>[7800],<Secondary__Default_IP>[7800]"
  7. Save the file. 
Secondary TrueSight Presentation Server


  1. Back up of the file %TRUESIGHTPSERVER_HOME%/truesightpserver/conf/ha/cache-replicator.conf and place the backup file in a different location.
  2. Open the file %TRUESIGHTPSERVER_HOME%/truesightpserver/conf/ha/cache-replicator.conf in edit mode.
  3. Find the line that starts with <TCP bind_addr="${jgroups.tcp.address:<Secondary__FQDN>}"
  4. Replace <Secondary__FQDN> with <Secondary__Default_IP>.
     After the changes the line appears as:
    <TCP bind_addr="${jgroups.tcp.address:<Secondary__Default_IP>}"
  5. Find line that starts with <TCPPING ergonomics="false" initial_hosts="<Secondary__FQDN>[7800],<Primary__FQDN>[7800]"
  6. Replace <Primary__FQDN> with <Primary__Default_IP> and <Secondary__FQDN> with <Secondary__Default_IP>
    After the changes the line appears as:
    <TCPPING ergonomics="false" initial_hosts="<Secondary__Default_IP>[7800],<Primary__Default_IP>[7800]"
  7. Save the file.

3. Start services on Primary node first, wait for it to be in Active              status and then start services on the secondary node.

4. Create new Integration Service clusters via the primary console.

5. Perform a failover to the secondary and now the Integration                Service clusters created in step 4 of primary console reflects in            the secondary console too.



Index Server does not start in Linux environments.


Error:

Failed to map segment from shared object: Operation not permitted.

It is a product requirement to provide the read, write, and executable permissions to systems tmp folder for user who installed or mount tmp directory without noexec options.

For more information, see 

Preparing to install the Presentation Server

If provided executable permissions on tmp directory is not possible then following steps can be performed as workaround:

  1. Create a custom tmp directory with executable permissions.
  2. Stop server using  the command 
    tssh server stop
  3. Navigate to %TRUESIGHTPSERVER_HOME%/truesightpserver/conf/services/idxsrv.conf file and
    vm.args.system.35=-Djava.io.tmpdir=<Custom_tmp_path>
  4. Navigate to %TRUESIGHTPSERVER_HOME%/truesightpserver/data/pgsql/postgresql.conf file
    unix_socket_directories = '<Custom_tmp_path>'
  5. Start TrueSight Presentation Server using the command
     tssh server start

FAQ

Is it possible to change the TrueSight Presentation Server hostname, FQDN, and domain after the initial TrueSight Presentation Server installation?

TrueSight Presentation Server does not currently support the change of hostname, FQDN, or domain name after installation.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*