Implementing private certificates in TrueSight Server Automation

Typically TrueSight Server Automation uses self-signed certificates to secure communication between clients and Application Servers. However, you might choose to provision Application Servers with a CA-issued certificate or certificate chain. This topic covers the following scenarios:

  • Replacing the existing certificate with a newly-generated self-signed certificate,
  • Obtaining a certificate signed by a trusted Certificate Authority (CA) – either a public CA or an internally-trusted CA for your organization.

This topic includes the following sections:

To generate a self-signed certificate for an Application Server

Performing this procedure generates a 2048-bit RSA key and a self-signed certificate for an Application Server. The certificate is valid for three years, and it is stored under the "blade" alias.

  1.  From <installDirectory>/bin, enter the following command: 
    blmkcert CN= <hostname> <jksFileName> <password>
    The command shown above has the following parameters:

    • <hostname> — Typically set to the host name where you are generating the certificate.
    • <jksFileName> — The full path to the keystore file that you are generating. This file will replace the existing keystore file in the deployments directory for the Application Server that is being updated, such as <installDirectory>/br/deployments.
    • <password> — A password used to encrypt the generated keystore file.

    For example, if you are generating a self-signed certificate on a Windows server called blapp1.example.com, you might enter a command similar to the following: 
    blmkcert CN=blapp1.example.com "bladelogic.keystore" password
    This will create a file named bladelogic.keystore in the current directory. It is not recommended to overwrite the existing bladelogic.keystore while the application server is running. 

    New in 8.9.03TrueSight Server Automation supports Secure Hash Algorithm 256 (SHA 256).

    For fresh installation of application server 8.9.03, the bladelogic.keystore is created with SHA256 Signature Algorithm.

    In case of upgrade, by default, the existing bladelogic.keystore file is not upgraded to SHA256. To use SHA256 in bladelogic.keystore for upgraded application server, perform the following steps:

    1. Back up the old bladelogic.keystore file.
    2. Recreate the bladelogic.keystore using blmkcert.
    3. Apply the same bladelogic.keystore across the MAS setups.
  2. After generating the new keystore file with the new certificate, skip to the Using the new keystoresection below.


Back to top

To secure communication with CA certificates

When you install an Application Server, the installation procedure provisions the Application Server with a self-signed certificate. However, some organizations might choose to use a CA-issued certificate rather than the default self-signed certificate.

When you perform this procedure, you set up a keystore that takes the place of the bladelogic.keystore created automatically when you install the Application Server. (A keystore contains certificates and a private key. A trust store only contains certificates.)

If the certificate you are importing includes a URL for an OCSP Responder, the client attempts to verify the revocation status of the Application Server certificate. If you do not want clients to verify a certificate's revocation status, do not provision the Application Server with a certificate that includes an OCSP URL.

After you provision Application Servers with CA-issued certificates, you should import those certificates into client trust stores. For more information about that procedure, see To import CA-issued certificates into clients.

Instructions for importing the certificate differ, depending on whether the certificate authority can generate the key and request or the certificate authority can only sign CSRs and does not have the ability to export a key.

To import a certificate when the certificate authority can generate the key and request

  1. Obtain the signed certificate and certificates for the CA chain from the certificate authority. The key must be generated with FIPS 140-2 compliant algorithms.
  2. Import the certificate and its corresponding private key into a keystore file on the Application Server. Your certificate authority may provide the certificate and key in a variety of formats.
    1. If the CA provides a JKS file you can skip to the Using the new keystore section below.

    2. If the CA provides a PKCS12 file  you can use the keytool command provided on the application server to convert to the JKS format by using the command below:

      <path to keytool>/keytool -importkeystore -srckeystore <pkcs12 keystore>
         -destkeystore <jks keystore> -srcstoretype pkcs12 -deststoretype jks
         -srcalias <pkcs12Alias> -destalias blade -destkeypass <password> -deststorepass <password>

      Ensure that you adhere to the following rules:

      • Use the same password for destorepass and destkeypass.
      • Use the alias blade for the certificate in the destination keystore.
      • Use the destination keystore type of JKS.

      In this command:

      • <path to keytool> will be inside the TrueSight Server Automation installation path. For example, C:\Program Files\BMC Software\BladeLogic\NSH\jre\bin on Windows or /opt/bmc/bladelogic/NSH/br/java/bin on UNIX), 
      • <pkcs12 keystore> is the file being imported.
      • <jks keystore> is the name of the keystore file you are creating. This file will be stored in the deployments directory for the Application Server that is being updated. For now specify a temporary location for this file.  Typically the file name is bladelogic.keystore.

      • <pkcs12Alias> is the alias under which the certificate and private key are stored in the PKCS12 store from your CA.  If you do not know this you can run the following command to obtain it:

        <path to keytool>/keytool -list -keystore <pkcs12 store> -storepass <pkcs12 password> -storetype pkcs12
      • <password> is the same password for both the keystore and key. The application server will be configured with this password via blasadmin in a later step.
  3. After converting the PKCS12 keystore to a JKS format, skip to the Using the new keystore section below.

Back to top


To import a certificate when the certificate authority only signs CSRs (but does not export a key)

  1. Run the following command to generate a new keystore using the keytool utility:

    <path_to_keytool>/keytool -genkey -alias blade -keyalg RSA -keystore <kestore filename> -storepass <password> -dname "CN=<FQDN>" -keypass <password> -validity <number of days> -keysize 2048 -sigalg SHA256withRSA -storetype jks

    Notes

    • The password for the keystore and keypass must be the same.
    • Use the alias blade for the certificate in the keystore.
    • Use the keystore type as JKS.

    In this command:

    • <path to keytool> will be a temporary location.
    • <keystore> is the file name of the keystore to be created.
    • <password> is the password for the keystore and key (they should be the same)
    • <hostname> is the hostname that you want to appear in the certificate.
    • <validity> is the length in days that the certificate is valid for.
    • <keysize> is the size/strength of the key. Typically this is 2048.

    Important

    A warning similar to the following one appears when you run the keytool command:

    The JKS keystore uses a proprietary format. It is recommended to migrate to PKCS12 which is an industry standard format using "keytool -importkeystore -srckeystore /bmc/install_files_temp/bladelogic.keystore -destkeystore /bmc/install_files_temp/bladelogic.keystore -deststoretype pkcs12".

    Ignore the warning. TrueSight Server Automation does not the support the PKCS12 format.

  2. Generate a certificate request from this keystore to send to the CA for signing.

    <path to keytool>/keytool -certreq -alias blade -ext SAN=DNS:appserver1.example.com,
    DNS:appserver2.example.com,DNS:appserver3.example.com -keystore <keystore file name>
    -file <csr file name> -storepass <password> -keypass <password>

    where,
    <csr> is the file name that will hold the Certificate Signing Request.
    SAN= contains the host name used in the CN at minimum, and all the Application Server names and any VIP name.

  3. Get the CSR signed by the CA and download the certificate (.pem or .crt file) with either the CA chain (typically a PKCS7, .p7b file) or a list of root, intermediary, and signing certificates (two, three, or more .pem or .crt files). If you get more than one file, add an additional -certfile argument in the openssl command. For example,

    openssl crl2pkcs7 -nocrl -certfile signed_cert.pem -out certs.p7b -certfile ca.pem -certfile intermediate.pem -certfile signing.pem

  4.  Import the .p7b file into the keystore.

    <path to keytool>/keytool -importcert -keystore <keystore> -file <p7b file>
       -storepass <password> -keypass <password> -alias blade -storetype jks

     In this command <p7b file> is the file name that contains the CA-signed certificate request and the CA chain.  The other values are from the previous step when you generated the new keystore.

  5. Now that you have imported the signed certificate and CA certs into the new keystore, goto the Using the new keystore section below.

    Back to top

Note

They keytool utility included with the Application Server should be used in all steps above that reference they keytool command. Ensure that you use a fully qualified path name when running the command, or ensure that you are using that keytool binary in the commands above and not some other keytool binary found in the system's path.

Using the new keystore


Once you have generated the new keystore file using one of the methods above you can start using it on your application server(s) by following these steps.

  1. Stop the Application Server.
  2. Make a backup of the existing <install dir>/br/deployments/bladelogic.keystore 
  3. Copy the new bladelogic.keystore file over the existing file in <install dir>/br/deployments

  4. If the keystore password used in the steps above is not the same as the one currently configured in the application server configuration, then you must update the application server configuration with the new password by running the following commands: 

    blasadmin -a set appserver certstore bladelogic.keystore
    blasadmin -a set appserver certpasswd <keystorePassword>
  5. If you have multiple Application Servers, repeat the above steps on those application servers. For information about this process, see Synchronizing keystore files of multiple Application Servers.
  6. Start the Application Server service.
    1. The first time that you connect to the application server from a RCP client you are informed that a new certificate has arrived from the Application Server. Accept the new certificate.  

    2. The following video demonstrates how to generate the certificate and synchronize it across Application Server deployments:

      icon-play2x.pnghttps://youtu.be/RHEn_86bk_4

Back to top

Troubleshooting issue with CA certificates

You might encounter the following issue during the import of CA certificates due to problems in the use of the keytool utility. Use the following troubleshooting steps to resolve the issue.

Symptom

Troubleshooting steps

The keytool utility is not responsive for a long time (typically more than 2 minutes) on a Linux application server

  1. Navigate to the java directories where the keytool utility is located, and locate the java.security file in /NSH/br/java/lib/security (on UNIX or Linux) or \NSH\jre\lib\security (on Windows).
  2. Open the java.security file using any text editor, and change the securerandom.source setting to the following value:
    securerandom.source=/dev/./urandom

To import CA-issued certificates into clients (optional)

If you have provisioned an Application Server to use a certificate or certificate chain obtained from a Certificate Authority (see To secure communication with CA certificates), you should import a related certificate into the client's trust store. The related certificate should be the issuing certificate for the Application Server certificate. If the Application Server is provisioned with a certificate chain, the certificate that you import into the client's trust store should be the issuing certificate for the top of the certificate chain.

If the above procedure is not performed, you will be prompted to accept the Application Server's certificate on first connection, just as you are prompted when you use the self-signed certificate that the Application Server is configured with by default. The benefit to importing the CA chain is that the client can perform the certificate validation and not prompt you with a certificate warning, so that you are sure that you are connecting to a validated Application Server. This is similar to visiting an HTTPS website that is signed by a CA in the browser's certificate store, as opposed to going to a website with a certificate signed by a CA that is not in the browser's certificate store, in which case you must manually inspect the certificate to ensure that you are connecting to the correct entity.

Use the blcred utility to import the certificate into the client trust store by entering the following command: 
blcred cert -import <certificateFile> 
In this command, certificateFile provides the path to the certificate you are adding to the trust store. This file must use the PEM or DER format. This file could be the Application Server certificate file, or it could be a CA's certificate that can be used to verify the validity of the Application Server certificate. OCSP verification on the client side only happens if the CA certificate was imported and the Application Server certificate contains an OCSP URL.

  • (Windows) The command imports the certificate to C:\Documents and Settings\user\Application Data\BladeLogic\client_keystore.pkcs12.PEM.
  • (UNIX) The command imports the certificate to userHomeDirectory/.bladelogic/client_keystore.pkcs12.PEM.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*