Converting to FIPS-140 mode
BMC recommends that you monitor the FIPS-140 mode conversion. See Monitoring FIPS-140 and normal mode conversions.
Perform the following steps for converting the FIPS-140 mode.
- Converting from BMC Atrium Single Sign-On to FIPS-140 mode
- Installing the unlimited strength policy files
- Installing the cryptography library
- Enabling FIPS-140 mode
- To deploy the web agent for FIPS-140
To convert from BMC Atrium Single Sign-On to FIPS-140 mode
When operating in FIPS-140 mode, BMC Atrium Single Sign-On blocks contact with products which are not also operating in a FIPS-140 compliant mode.
Before performing the switch to FIPS-140 mode
- Perform a system backup before switching to (or from) FIPS-140 mode. An unexpected hardware or software failure during the conversion can corrupt the server configuration.
- Verify that the integrated BMC products are capable of operating in a FIPS-140 compliant mode and are capable of making the reconfiguration that is required to continue operating with BMC Atrium Single Sign-On.
- If you plan to integrate additional products with BMC Atrium Single Sign-On after the switch to FIPS-140 mode is complete, be sure that these products can be integrated with the server. See the BMC Atrium Single Sign-On Product Availability Compatibility on the support website.
- Ensure that your Internet browser is capable of supporting 256-bit Advanced Encryption Standard (AES) encryption. See #Browser cipher capabilities.
- Obtain the RSA CryptoJ FIPS cryptography module. See #RSA CryptoJ FIPS cryptography module.
Contact Customer Support for access to the RSA CryptoJ FIPS cryptography module. This library file must be installed into the server's Java Virtual Machine (JVM), replacing the current version which is not certified.
Obtain unlimited strength Java policy files.
BMC Atrium Single Sign-On uses Oracle JVM 1.7.0. The unlimited policy files for this JVM are available for download from the following URL: http://java.sun.com/javase/downloads/index.jsp.
Browser cipher capabilities
When operating in FIPS-140 mode with default networking ciphers, the Internet browser must be capable of supporting 256-bit Advanced Encryption Standard (AES) encryption. Otherwise, the browser cannot connect with BMC Atrium Single Sign-On for administrator or user authentication purposes. FireFox 3+ is able to operate at this level. Internet Explorer might not be able to support 256-bit AES depending on the version.
You can check your browser cipher capabilities at the following URL: http://www.fortify.net/sslcheck.html. This web site provides the encryption status of your browser.
RSA CryptoJ FIPS cryptography module
The FIPS-approved cryptography module used by BMC Atrium Single Sign-On for FIPS-140 compliance is the RSA CryptoJ library version 6.1.
The following table shows the algorithms used in normal mode and FIPS-140 mode.
MD5, SHA1, SHA256, SHA512
SHA1, SHA256, SHA512
TLS_RSA_WITH_AES_256_CBC_SHA, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P256, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P384, TLS_ECDHE_RSA_WITH_AES_256_CBC_SHA_P521, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P256, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P384, TLS_ECDHE_ECDSA_WITH_AES_256_CBC_SHA_P521, TLS_DHE_DSS_WITH_AES_256_CBC_SHA
To install the unlimited strength policy files
BMC Atrium Single Sign-On uses Oracle JVM version 1.7.0. By default, this JVM is installed with strong encryption policy files allowing for limited strength settings for encryption algorithms. These limitations prevent BMC Atrium Single Sign-On from running in FIPS-140 mode. To overcome this limitation, the Unlimited Strength Jurisdiction Policy Files must be downloaded from Oracle and installed into the BMC Atrium Single Sign-On JVM.
BMC Atrium Single Sign-On and all integrated products must be shut down before installing the unlimited strength policy files. BMC Atrium Single Sign-On cannot be in use during the conversion to FIPS-140 mode. If possible, a firewall should be employed to block all remote access to the server.
- Shut down all BMC Atrium Single Sign-On integrated products.
- Stop BMC Atrium Single Sign-On.
- If you have not done so already, download the archive that contains the unlimited strength policy files from the following URL: http://java.sun.com/javase/downloads/index.jsp.
- Extract the contents of the files.
- Make a backup copy of the currently installed strong strength policy files.
- Copy the unlimited strength policy files into the BMC Atrium Single Sign-On JVM.
The JVM is located in the following default location:
- (Microsoft Windows) C:\Program Files\BMC Software\AtriumSSO\jre\lib\security
- (UNIX) /opt/bmc/AtriumSSO/jre/lib/security
If BMC Atrium Single Sign-On has been installed in a non-default location, the location of the JVM can be determined by using the following pattern:
- (Windows) <installationDirectory>\AtriumSSO\jre\lib\security
- (UNIX) <installationDirectory>/AtriumSSO/jre/lib/security
In this case, installationDirectory is the base directory selected during the server installation.
For BMC Atrium Single Sign-On servers using an external Tomcat server, the location of the JVM was determined by the administrator that configured the Tomcat server. Regardless of the JVM location, the following templates indicate the correct location:
- (Windows) <jdkDirectory>\jre\lib\security
- (UNIX) <jdkDirectory>/jre/lib/security
In this case, jdkDirectory is the base directory of the JDK used to run BMC Atrium Single Sign-On.
To install the cryptography library
For cryptographic functions in normal mode, BMC Atrium Single Sign-On uses the JVM and a version of the RSA CryptoJ library that is not certified for FIPS-140 operation. However, when placed into FIPS-140 mode, the server reconfigures the JVM to use the RSA CryptoJ provider as the primary provider. In addition, the cryptography needs of the server exclusively uses this provider.
For the server to start in FIPS-140 mode successfully, the FIPS-140 certified version of the RSA CryptoJ library must be installed into the JVM, replacing the uncertified version. The versions of the library can be externally identified by the names of the libraries. Normal mode library is cryptoj.jar and the FIPS-140 mode libraries are cryptojce.jar, cryptojcommon.jar, and jcmFIPS.jar.
Contact BMC Software support for instructions on accessing the FIPS-140 version of the library.
- Make a backup copy of the cryptoj.jar file. You might need to restore BMC Atrium Single Sign-On to normal encryption mode.
- Copy the FIPS-140 mode libraries are cryptojce.jar, cryptojcommon.jar, and jcmFIPS.jar files onto the file system of the computer hosting BMC Atrium Single Sign-On.
- Copy the FIPS-140 mode libraries are cryptojce.jar, cryptojcommon.jar, and jcmFIPS.jar files to the server's JVM library directory.
Remove the cryptoj.jarfile.
This is an important step to prevent a collision of the two libraries.
JVM library file location
The JVM library is located in the following default location:
- (Microsoft Windows) C:\Program Files\BMC Software\AtriumSSO\jre\lib\ext
- (UNIX) /opt/bmc/AtriumSSO/jre/lib/ext
If BMC Atrium Single Sign-On server has been installed in a non-default location, determine the location of the JVM library using the following pattern:
- (Windows) <installationDirectory>\AtriumSSO\jre\lib\ext
- (UNIX) <installationDirectory>/AtriumSSO/jre/lib/ext
In this case, installationDirectory is the base directory selected during the server installation.
For BMC Atrium Single Sign-On servers utilizing an external Tomcat server, the location of the JVM was determined by the administrator that configured the Tomcat server. Regardless of the JVM location, the following templates indicate the correct location:
- (Windows) jdkDirectory\jre\lib\ext
- (UNIX) jdkDirectory/jre/lib/ext
To enable FIPS-140 mode
After restarting BMC Atrium Single Sign-On with the required JVM modifications in place, the server's configuration can be updated to trigger the change of cryptography. Before performing this next step, be sure that the following JVM modifications have been performed:
- Unlimited strength policy files are installed.
- The library cryptojce.jar, cryptojcommon.jar, and jcmFIPS.jar files are installed in library directory.
- The library cryptoj.jar file has been removed from the library directory.
- (Optional) Update your network ciphers if desired. See Changing FIPS-140 network ciphers.
- Restart BMC Atrium Single Sign-On.
- Log on to BMC Atrium Single Sign-On administrator console.
- Click Edit Server Configuration.
- Select Enable FIPS-140
After the configuration has been successfully saved, the conversion process starts. This process cannot be interrupted. Do not stop BMC Atrium Single Sign-On, log on with another Administrator console, log off the current Administrator console, or initiate any other interactions with the server.
This process takes approximately 10 to 20 seconds, depending upon the computer hardware. Be sure that the background task validation process posts a successful conversion message before proceeding to the next step.
- Monitor the log files for the completion of the cryptography conversion. For more information on how to monitor the conversion, see Monitoring FIPS-140 and normal mode conversions.
- After the conversion process completes, stop and start the server.
- Verify that the server is properly operating in FIPS-140 mode by viewing the BMC Atrium Single Sign-On log file (for example, atsso.0.log)
Reconfigure all integrated products to operate in FIPS-140 mode.
All products which were configured with BMC Atrium Single Sign-On prior to conversion to FIPS-140 mode must be reconfigured to operate in FIPS-140 compliant mode. These integrated products cannot use BMC Atrium Single Sign-On for authentication until they are synchronized with BMC Atrium Single Sign-On.
To deploy the web agent for FIPS-140
If a product is already integrated with the BMC Atrium Single Sign-On server when the BMC Atrium Single Sign-On service is switched to FIPS-140 mode, follow these steps to switch the agent to FIPS-140 mode:
- Modify the integrated BMC application to run in FIPS-140 mode.
Please check with the BMC application support team for the instructions on how to enable FIPS-140 mode within the application itself.
- Stop the integrated BMC application server.
- Open the cfg.properties file stored at <Web Server Home>\atssoAgents\test\cfg.properties location and verify that fipsMode parameter's value is set to true.
- Start the integrated BMC application again.
The following steps are used to integrate a new BMC application with an existing BMC Atrium Single Sign-On server which is already running in FIPS-140 mode. Follow these steps to modify the JVM that will be used to execute the deployer utility for the integration task.
The following steps do not change the FIPS-140 mode of the BMC application itself. Please check with the BMC application support team for the instructions on how to enable FIPS-140 execution within the BMC application itself.
- Install the unlimited strength policy files and cryptography libraries as mentioned in the earlier steps.
Replace the existing list of security providers in the java.security file stored at the JAVA_HOME_FOR_AGENT>\jre\lib\security\ location with the following list of providers from the java.security.fips file.
# List of providers and their preference orders (see above): # security.provider.1=com.rsa.jsafe.provider.JsafeJCE security.provider.2=sun.security.provider.Sun security.provider.3=sun.security.rsa.SunRsaSign security.provider.4=sun.security.ec.SunEC security.provider.5=com.sun.net.ssl.internal.ssl.Provider security.provider.6=com.sun.crypto.provider.SunJCE security.provider.7=sun.security.jgss.SunProvider security.provider.8=com.sun.security.sasl.Provider security.provider.9=org.jcp.xml.dsig.internal.dom.XMLDSigRI security.provider.10=sun.security.smartcardio.SunPCSC security.provider.11=sun.security.mscapi.SunMSCAPI com.rsa.cryptoj.jce.kat.strategy=on.load com.rsa.cryptoj.jce.fips140initialmode=FIPS140_SSL_MODE
Run the deployer.jar.
If you running the deployer on JDK or JRE version 1.7.0_51 or later, you must use the JVM parameter, -Dhttps.protocols=TLSv1.2.
"C:\Program Files\Java\jdk1.7.0_51\bin\java.exe" -Dhttps.protocols=TLSv1.2 -jar deployer.jar --install --container-type tomcatv6 --atrium-sso-url https://sso-server.bmc.com:8943/atriumsso --web-app-url https://sso-server.bmc.com:8943/test/index.html --container-base-dir "C:/tomcat6" --admin-name amadmin --admin-pwd admin123 --truststore "C:/tomcat6/conf/cacerts.p12" --truststore-type PKCS12 --truststore-password changeit --jvm-truststore "C:/Program Files/Java/jdk1.7.0_51/jre/lib/security/cacerts" --jvm-truststore-password changeit
After the agent deployment to the FIPS enabled server, open the cfg.properties file stored at <Tomcat>\atssoAgents\test\cfg.properties location and verify that fipsMode parameter's value is set to true.
Modify the network ciphers. For information about the network ciphers, see Changing FIPS-140 network ciphers.
- After saving the changes, restart the container server.
The web agent updates the encryption level and integration with the BMC Atrium Single Sign-On server.