Troubleshooting certificate issues

When the required Certificate Authority (CA) certificates are missing, incorrectly imported, or not trusted in BMC Helix Innovation Suite, login failures, integration issues, or pod communication errors might occur.

Issue symptoms

Certificate-related errors occur in BMC Helix Innovation Suite due to failures in certificate trust validation.

• Certificate not trusted errors
• Login failure after certificate replacement
• Integration failures between services

The issue usually occurs in below areas:

• BMC Helix Innovation Suite platform pod
• Java truststore (cacerts)
• During ingress certificate updates
• During communication between platform components

Issue scope

This issue occurs under the following conditions:

• After replacing or renewing Ingress certificates
• When the Certificate Authority (CA) has changed
• When the Java truststore (cacerts) does not contain the required CA certificate
• When configmaps are not updated with both old and new CA certificates before certificate rotation

Resolution

Perform the following steps to verify and update Certificate Authority (CA) certificates for the BMC Helix Innovation Suite platform.

1. To view the BMC Helix certificate details, run the following command:
   
   openssl s_client -showcerts -connect <HELIX_HOST>:443 </dev/null  | openssl x509 -noout -text
   
   Replace <HELIX_HOST> with the Helix hostname you want to verify, for example, the Helix Platform LB_HOST or Helix IS mid-tier

2. To retrieve the cacerts file from the Service Management namespace, run the following command:
   
   kubectl get secret -n <HELIX-IS-NAMESPACE> cacerts -o jsonpath='{.data.cacerts}' | base64 -d > cacerts

3. To verify that the expected certificate(s) are present at the following location:
   
   /path_to_java/bin/keytool --list -keystore ./cacerts -alias <ALIAS_NAME>
   
   Replace <ALIAS_NAME> with the alias used when importing the certificate(s).

4. (Optional)To validate the cacerts file using SSLPoke, run the following command:curl -s https://gist.githubusercontent.com/MatthewJDavis/50f3f92660af72c812e21b7ff6b56354/raw/SSLPoke.java -o SSLPoke.java
   javac SSLPoke.java
   java -Djavax.net.ssl.trustStore=./cacerts SSLPoke <HELIX_HOST> 443

    

5. If the CA certificates are missing or outdated, import the required certificate by running the following command:

    

   keytool -importcert -v -alias <ALIAS_NAME> -file <CERTIFICATE_FILENAME> -keystore ./cacerts

6. Repeat the SSLPoke validation described in step 4 to confirm that the cacerts file contains the required certificates.

7. To recreate the cacerts Kubernetes secret with the updated cacerts file, run the following command:
   
   kubectl delete secret cacerts -n <HELIX-IS-NAMESPACE>
   kubectl create secret -n <HELIX-IS-NAMESPACE> generic cacerts \
   --from-file=cacerts \
   --dry-run=client -o yaml | kubectl apply -f -

8. Restart the BMC Helix Innovation Suite pods to load the updated truststore.

9. Verify that all pods restart successfully and that BMC Helix Innovation Suite services are accessible without SSL errors.