Enhanced encryption installation procedure

1. Log in to the platform running the main BMC Defender Server by using an administrative login.
2. Copy the Enhanced encryption software package on to the platform. Verify that this is the precise package obtained from BMC software. (If necessary, you can use the MD5 signature for the software package, obtained from BMC Software.) The name of this package is co-N-N-N-tls.exe, where N-N-N is the version number for the package.
3. Run the package. The package displays the version number and builds date for the software.

4. Click OK to close the version number dialog box, and then click Unzip to unzip files to the BMC Defender Server root directory.
   

   Warning

   Notes

   • Before unzipping files, adjust the location of the Unzip to folder value to be the precise location where BMC Defender Server is currently installed. The actual location might vary depending on the location at which the BMC Defender Server was originally installed. 
   • You can determine the install directory for BMC Defender Server by using the web interface, using the More > Sys Info menu from the upper-right corner of the BMC Defender Server web display.
   • If you are unable to extract files and receive a message indicating that the CO-apache-tls.exe program (or other program) is busy. This is because you are reinstalling the program and the CO-apache-tls.exe program is busy. In this case, stop the BMC Defender Apache TLS service via the Windows Service Manager and then extract files.
5. After extracting files, the Windows Setup Wizard automatically starts. Click Next to go to the next window.

6. In the second window of the Setup Wizard, you receive a cipher key seed. Enter a random string ranging from 8 to 32 characters. This key forms the basis of the encryption that is unique to this BMC Defender Server site. You cannot store the key, and you do not need to recover or remember it. The value is strictly used to ensure a highly random encryption key for the message encryption.

   Warning

   Note

   The key must contain at least one upper-case letter, one lower-case letter, one punctuation mark, and one number. You can typically type letter keys and numbers at random, holding down and releasing the Shift key needed to ensure a variety of characters. The Setup Wizard checks the strength of the cipher key seed and does not permit you to continue if the key is not sufficiently random.

7. In the third window of the Setup Wizard, enter the port number for the HTTPS server. To use the standard port number, enter a value of 443.

   Warning

   Note

   The Setup Wizard selects a number for you based upon the available free service ports on the system. The specified service port must be free from other programs. The Setup Wizard verifies that the port number is available and does not permit you to continue if the port number specified is currently in use.

8. In the fourth window of the Setup Wizard, enter the identity information needed to create the security certificate for the Apache server. You can use the defaults or enter a different company name, email address, and website server name.

   Warning

   Note

   The Common Name must be the name of the device used in the URL when accessing the agent. This might not be so if you access the BMC Defender Server using a name other than the configured host name.

   If the host name for the server is BMC Defender Server, but the official DNS name is www.bmc.com, then you might receive a warning about the certificate when accessing the platform.

9. In the fifth window of the Setup Wizard, you receive a prompt to continue. If you find an existing certificate in the ssl folder, you must check a box to overwrite that certificate information. To close the Setup Wizard, click Finish. The Apache TLS Server is automatically installed and starts. The Enhanced Encryption software gets ready for configuration.

Installation checkout and verification

You should be able to access the BMC Defender Server using HTTPS rather than HTTP immediately after installing the BMC Defender Server software. When you can specify the URL for the BMC Defender Server, the website logon window is displayed.

If you cannot access BMC Defender Server by using the HTTPS URL, then the BMC Defender Apache TLS Adapter might not be properly installed and start. You can troubleshoot this problem as follows:

1. Verify that the BMC Defender Apache TLS Adapter entry exists in the Windows Control Panel > Admin Tools > Services window. If this entry does not exist, the service installation failed. Contact BMC Support for assistance.
2. Verify that the BMC Defender Apache TLS Adapter gets properly started. Run the Windows Task Manager. The CO-apache-tls.exe program should appear as a running process. If this process is not running, change working directories to the apache-tls\bin directory and try executing the CO-apache-tls.exe program at a command prompt. Inspect the command output for obvious permission errors.
3. Use the netstat –a –n –p tcp program at a command prompt and verify that the service port specified in the third Setup Wizard window is listening for requests. If the port number is listening and the CO-apache-tls.exe program is running, then a firewall or proxy issue is preventing access to the BMC Defender Server program. Review this problem with network administrators at your site.
4. Inspect the logs\error.log file for error messages. Contact BMC Support for assistance, and be prepared to send this log file for analysis as needed.
5. Verify that the Windows Firewall permits access to the configured TCP port for the server, by default the TCP 443 (but possibly some other port number configured during installation).