Troubleshooting BMC Digital Workplace Catalog
The topic provides information about how to troubleshoot the most common issues related to BMC Digital Workplace Catalog.
This topic covers the most common issues ranging from installation and upgrade to web server (Jetty) to process logging. The sections below discuss each issue in detail.
Diagnosing and reporting an issue
Instructions: After you identify the symptoms and scope of the issue, use this troubleshooting guide to diagnose and resolve the issue or to create a BMC Support case. To see the contents of BMC Digital Workplace Catalog files, see Contents of the logs.
|Installation and upgrade|
The installation script informs you if there are any problems. Full details are available in the arsystem_install log, which is located in the /tmp directory.
To obtain a live feed on installation progress, run the
The following extract is an example of a successful installation.
If the installer fails and the arsystem log doesn't identify the root cause, run the uninstall script (make sure you also remove any BMC Digital Workplace Catalog database references, if present) and run the script again in verbose mode as follows:
This command generates a detailed log to the terminal. In the event of a failure, the output will stop allowing you to scroll up and identify what was happening at the time.
Back up the BMC Digital Workplace Catalogdatabases and file system before upgrading. If you encounter a problem, you can roll back and send the logs to BMC Technical Support.
If you receive error or warning messages during installation, see Troubleshooting installation and upgrade issues in BMC Digital Workplace.
Stopping and starting BMC Digital Workplace Catalog
Use the dwpcontroller script from the $AR_SYSTEM_HOME directory to restart BMC Digital Workplace Catalog. (This script also restarts the platform.)
|RSSO logs and diagnostics|
To write an RSSO debug to the bundle log, add the following lines under the existing bundle section.
To enable RSSO logging:
In version 19.02 and later, a diagnostic tool checks whether the RSSO configuration is correct. (This tool is located in the $AR_SYSTEM_HOME/sb/configure_rsso directory.) To run open the run_diagnostic.sh script and make sure the paths and diagnostic file name are correct, run the following command:
After the run_diagnostic.sh script is run, the output should look something like this:
|Web server (Jetty)|
Jetty logging can be very useful to capture HTTP traffic.
To enable Jetty logging, uncomment the following section and change the log level:
If connection issues are observed in a server group, run the following command to analyze the traffic (where
Connector logging is available in version 19.02 and later.
To enable connector logging, change the log level to
You can also test the connector from the Connector Management page in BMC Digital Workplace Catalog.
The following screenshot indicates that the BMC Digital Workplace Advanced ITSM Integration patch needs to be updated.
Back up logback_server.xml and add the following to the end of the file:
Then log into the mid-tier (Catalog) as dwpadmin and navigate to Sever Information > Log Files and tick "Process Log".
You need to perform a "dwpcontroller stop/start" for the changes to take effect. Please remember to turn off this logging if it's no longer required.
BMC Digital Workplace bundle log
Almost every troubleshooting scenario will require analysis of the bundle log. Here's an example of what a "healthy" bundle log looks like:
In this example, DEBUG is enabled and it's constantly looping to check each connector's status.
In the following screenshot, Allen submitted a Service Request in BMC Digital Workplace, the Status is Fault.
To get a live feed of what's happening in the log, open a session in the $AR_SYSTEM_HOME/db directory, and run a tail on the log:
In the following example, the service request is faulting because a required field is empty:
In the following example, another catalog request is submitted from the BMC Digital Workplace end user console, and the bundle log shows that a Work Order Manager should be defined in Remedy ITSM:
Most errors are now visible by simply opening the Request from the Console.
To learn more about how to configure the logs and troubleshoot, see the following topics:
Logging level is controlled within logback_server.xml, which is located in the $AR_SYSTEM_HOME/conf directory.
Note: Before making any changes, back up logback_server.xml. XML mistakes may impact BMC Digital Workplace Catalog on startup.
The best level for debugging is level=all, but it can impact performance.
In 19.02 and later versions, you can control BMC Digital Workplace Catalog logging from the Application Settings panel.
Approvals and Remedy ITSM integration
The MyITSB-Approval process sends approval requests from BMC Digital Workplace Catalog to the Remedy ITSM server. When a service request is raised from the BMC Digital Workplace end user console:
If the correct approval status is not reflected in BMC Digital Workplace, enable filter/API logging on the Remedy ITSM server before reproducing the problem. Then, you can see the relevant filters triggering the approval change and sending it back to BMC Digital Workplace Catalog.
Then, open the SB:ServiceRequestStub form on the Remedy ITSM erver and select the Error Message check box. The following example shows a timeout error reaching BMC Digital Workplace Catalog:
Make sure the SB:LocalApprovalConfiguration and SB:RemoteApprovalConfiguration Approval forms on the Remedy ITSM servers have the correct information.
To track the status transmission back to BMC Digital Workplace Catalog, use the remoteaction log file (enabled in the logback_remoteaction file).
BMC Digital Workplace Catalog debug
Open the logback-dwp.xml file from the /opt/apache/tomcat8.x/external-conf directory and change these two extracts:
After the scan period of 60 seconds, a new log file (dwp-catalog.log) is created in the BMC Digital Workplace logs directory.
The following illustration depicts the process flow when submitting a request from BMC Digital Workplace. Approvals and Remedy ITSM use a combination of the remoteaction.xml and associated JAR files in combination with Remedy workflow, so you can use the same troubleshooting methodology as described above for approvals.
The Status Change filters have changed names over the releases.
Catalog File System
Most of the BMC Digital Workplace Catalog files are contained within the digitalworkplace directory. Some of the more important directories and files are highlighted in the following illustration:
BMC Digital Workplace Catalog is run through a variety of bundles, which are loaded after the dwpcontroller script is run as part of a platform restart. Unlike the BMC Digital Workplace client, you cannot restart the individual components of BMC Digital Workplace Catalog (without negative consequence); however, you can check the bundle status of BMC Digital Workplace Catalog by using the Eclipse Open Services Gateway Initiative ( OSGI) Console.
The logging level for OSGI is controlled in the $AR_SYSTEM_HOME/conf/config.properties file:
# Enable osgi console for debugging
osgi.console = 12666
To connect to the console, use the
telnet command to connect to the BMC Digital Workplace Catalog host and execute the
[root@server]# telnet localhost 12666
You can identify important bundles related to BMC Digital Workplace Catalog by looking for the ones with
If you notice any bundles in a RESOLVED state, you can attempt to restart with the following commands:
If the BMC Digital Workplace Catalog login page becomes unavailable for all users, restart com.bmc.myservice.uiservice-war_0.0.1 and try again. If you are still unable to log in, contact BMC Support. (Enter
disconnect to end the session.)
The connectors and bundles are located in the $AR_SYSTEM_HOME/deployedsmart bundles directory.
DWP Controller Scripts
The rxscripts directory and subdirectories contain scripts that BMC Digital Workplace Catalog calls. In some cases, you may also run them manually.
|/bin/create_schema.sh||Gets called by the post install script|
|/bin/current-user.sh||Checks the current user who’s in the current session|
|/bin/tenant/find.sh||Lists all the tenants|
|/bin/tenant/delete.sh||Deletes a tenant|
|/bin/connector/refresh.sh||Refreshes the available connectors|
Checks if BMC Digital Workplace Catalog is up or down
|/bin/login.sh||Used to log in to Remedy from the command line|
|/bin/run_migrations.sh||Gets called when database migrations are required|
|/bin/setenv.sh||Sets up the environmental variables to allow the API endpoints|
|/bin/tenant_seed.sh||Gets called by the post install script|
|/bin/version.sh||Gets the current version by running|
Lists all catalog services visible by the current BMC Digital Workplace user
Removes and finds workflow
To run these scripts, set up your environmental variables by sourcing the setenv.sh script:
[root@server bin]# source setenv.sh
After you source the setenv script, log in to BMC Digital Workplace Catalog from the command line and generate an authentication cookie. After the mycookies key is generated, subsequent scripts run (provided they are in the same directory as mycookie.txt).
[root@server bin]# ./login.sh email@example.com password
[root@server bin]# ls -lrt
-rw-r--r-- 1 root root 610 Jun 15 14:59 mycookies.txt
In the following example, the find.sh script is run to obtain information about a problematic tenant. You can also delete and re-create a tenant by using the post-install scripts (be careful deleting files).
[root@server bin]# tenant/find.sh
[root@server bin]# tenant/delete.sh coffee
You can enable Developer Mode (F12) in a browser. Then, open the Network tab response and look for any exceptions. The following screenshot illustrates how the root cause of a problem was captured by simply clicking the REST call and selecting the response.
Ensure BMC Digital Workplace Catalog is reachable and the fully qualified domain name is being used.
Users must exist on both of the following forms:
- User form on the BMC Digital Workplace Catalog server (which is populated via the sync script)
- CTM:People form in Remedy ITSM