Troubleshooting authentication issues
This topic provides information for troubleshooting authentication issues of Remedy Single Sign-On (Remedy SSO).
Generic authentication issues
Unable to define authentication chain for the client
When the user request is redirected to Remedy SSO login URL, the message 'Could not define authentication chain for the tenant:*' is displayed. This happens when the administrator deletes the default realm '*', adds another realm, and does not configure a domain for the new realm.
Workaround: Do one of the following:
- Add default realm back with '*' as realm ID.
- Add the BMC Remedy Mid Tier host name or FQDN in the realm domain if the realm id is not '*'.
Remedy AR authentication issues
Remedy SSO AREA plugin error after Remedy SSO
The user completes Remedy SSO login (either on the Remedy SSO login page for AR authentication or on the IdP login page) and is redirected back to BMC application URL. Then, the application may display an authentication error. The AR AREA plugin log file (<AR>/Arserver/Db/arjavaplugin.log) might contain the following or similar error logs.
2015-09-13 17:04:21,324 ERROR [pool-4-thread-10] com.bmc.arsys.pluginsvr.plugins.ARPluginContext (?:?) - <ARSYS.AREA.RSSO>Could not validate userId with Service Provider. Could not retrieve user from authentication string. 2015-09-13 17:04:21,324 ERROR [pool-4-thread-10] com.bmc.arsys.pluginsvr.plugins.ARPluginContext (?:?) - <ARSYS.AREA.RSSO>Return Code:2
Workaround: To troubleshoot this error, perform the following steps.
- Check the file contents of '<AR>/Conf/rsso.cfg'. Ensure the value of 'sso-service-url' property is the Remedy SSO server URL, and it is accessible from the AR server node.
- You can use curl to test network connectivity from AR server node to Remedy SSO server URL.
- If connection test is OK, use curl go to the next step.
- Check if any proxy is used on AR Plug-in server for outgoing network connection.
- The proxy option must be defined in the Java options in armonitor.cfg: the startup string for the java plugin server would have Java params for enabling proxy.
- If proxy server is defined in Java plugin Server, force bypass of proxy for Remedy SSO server host name.
No groups for authenticated users
When AR 8.x is integrated with Remedy SSO and if authenticated users have no groups after login, uncomment the following setting in the <AR>/Conf/rsso.cfg file:
SAML authentication issues
IdP error on SAML request if SAML IdP login URL contains a query parameter
If the IdP login URL contains a query parameter, for example if you see '?' in the URLyou might be displayed an error when the browser is redirected to IdP login URL. Hence as a user you cannot continue the login process.
SAML IdP returns NameID with encrypted string
Some IdPs might return an encrypted string in the NameID of the response.
Workaround: If the NameID in a SAML response returned by IdP exceeds 255 characters, increase the size of the NameId column in the IssuesTokens table of the Remedy SSO database. For example, if the NameID length is 300 characters, set the NameId column to at least VARCHAR(300).
Kerberos authentication issues
- Invalid keytab index number for Kerberos
- Browser sending NTLM instead of Kerberos
- Keytab file does not contain an entry to decrypt a service ticket
- Debugging Kerberos authentication issues
- Kerberos related events and messages
- Error: Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96 (or AP REP - AES128 CTS mode with HMAC SHA1-96)
- Increasing HTTP header size in Tomcat
Invalid keytab index number for Kerberos
This exception failure is generated in the logs when the keytab file is generated with a key version number (KVNO) number different from the one specified in the ticket. The solution is to regenerate the keytab file. Be sure to specify the /kvno 0 option; this ensures that the KVNO value is compatible.
amJAAS:10/18/2011 09:35:00:435 AM PDT: Thread[http-8443-1,5,main] Exception: com.sun.identity.authentication.spi.AuthLoginException: Failed to authentication. Failure unspecified at GSS-API level (Mechanism level: Specified version of key is not available (44))
Browser sending NTLM instead of Kerberos
Ensure that Remedy SSO server host name or domain is added in the list of websites for Kerberos authentication.
The entry "Authentication token is NTLM but not SPNEGO" in the log file indicates that the token that Remedy SSO sever receives from the client is a Microsoft Windows NT LAN Manager (NTLM) token and not a Kerberos token as required. The failure could be caused for the following reasons:
- Browser is not correctly configured to use Kerberos authentication. Refer Configuring the browser.
- Problem related to obtaining a service token for the Remedy SSO server. Make sure that Kerberos Service ticket is obtained when it tries to get an access to Remedy SSO server on client's machine, it should look as something like the below where
is the hostname of the machine with Remedy SSO Server (this check guarantees that customer's machine asked it from KDC and it then issued the service ticket).
c:\Windows\system32\klist.exe ... #1> Client: dev_adei @ QA2R.LOCAL Server: HTTP/access.bmc.com @ QA2R.LOCAL KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96 Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize Start Time: 11/3/2016 9:00:43 (local) End Time: 11/3/2016 19:00:43 (local) Renew Time: 11/10/2016 9:00:43 (local) Session Key Type: AES-256-CTS-HMAC-SHA1-96 ...
Keytab file does not contain an entry to decrypt a service ticket
If you get the following events, it indicates that the keytab file does not contain an entry to decrypt a service ticket.
java.security.PrivilegedActionException: GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed) Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed) Caused by: KrbException: Checksum failed Caused by: java.security.GeneralSecurityException: Checksum failed
Examine items and make sure that SPNs are valid. In case if SPN password is used in the Remedy SSO Admin console, ensure that the "Service Principal Name" is specified as
To see content of the existing keytab file, run the ktab utility as follows:
c:\java sun.security.krb5.internal.tools.Ktab -l -e -t -k all.keytab Keytab name: all.keytab KVNO Timestamp Principal ---- ----------------- ------------------------------------------------------------------------------------ 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (1:DES CBC mode with CRC-32) 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (3:DES CBC mode with MD5) 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (23:RC4 with HMAC) 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (18:AES256 CTS mode with HMAC SHA1-96) 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (17:AES128 CTS mode with HMAC SHA1-96)
Note, that this keytab file contains five entries for the same principal, but each entry has a different encryption type. You must use the option "/crypto all" with the ktpass utility to generate the keytab file.
Debugging Kerberos authentication issues
You can perform the following tasks while debugging Kerberos authentication:
Run the following command to ensure that customer's machine has joined the domain and a domain user is used:
- Ensure that the customer has other internal resources with Kerberos authentication and the customer can successfully log into them and use them from his machine. For this, you must have service tickets for them in the output of the klist utility.
- Ensure that Remedy SSO server is configured to use the same domain that the customer machine has joined.
- Ensure that that you are trying to access the Remedy SSO server using its FQDN, for example, http://access.example.com/rsso/. Also, the host name used in the FQDN is identical to the host name used in SPN for a service account created in Key Distribution Center. In this case, the SPN will be HTTP/access.example.com.
- Ensure that the Kerberos TGT ticket is obtained on the customer machine.
- Ensure that the browser is configured properly according to steps described in Kerberos authentication process.
- Ensure that the KDC domain is defined in uppercase in the Remedy SSO Admin console.
- Ensure that the time difference between KDC and the customer machine is no more than 5 minutes.
- Ensure that the Kerberos Service ticket obtained on the customer machine accessing the Remedy SSO server looks like HTTP/access.bmc.com@RSSO.COM where where access.bmc.com@RSSO.COM is the host name of the machine that hosts the Remedy SSO server.
Kerberos related events and messages
Ensure that the Remedy SSO logging level is set to TRACE.You can find the Kerberos related events and log information in the following files that are normally located in the log directory of Tomcat.
- rsso.log: Main log file for Remedy SSO server.
- tomcat8-stdout.*.log: Kerberos related events from Java Authentication and Authorization Service that is used by Remedy SSO server internally to authenticate users.
Error: Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96 (or AP REP - AES128 CTS mode with HMAC SHA1-96)
If you find this error in the rsso.log file, you have to install JCE Unlimited Strength Jurisdiction Policy Files for JDK/JRE to support AES128 and AES256 encryption types. You can find the policy files at the following links:
- Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files for JRE 8
- Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files for JRE 7
Also, you need to install these policy files in JRE that is used by the Tomcat on which the Remedy SSO runs
Increasing HTTP header size in Tomcat
The Kerberos service ticket is passed as a header value in the http request. Though the default maximum header size in tomcat is 4096 (4KB), under some circumstances the header size may go up to 28 KB. The login fails because of the large size, and the browser displays an error message because Tomcat does not respond to such requests.
To fix this issue, specify a maxHttpHeaderSize attribute on the https connector and set a large enough value in bytes.
LDAP authentication issues
LDAP authentication failure
When using LDAP (LDAP over SSL) as the authentication method in an environment that uses Java 7.75+ / Java 8.31+, users randomly are not authenticated with the following exceptions:
simple bind failed: <LDAP SERVER NAME>:PORT [Root exception is
javax.net.ssl.SSLHandshakeException: server certificate change is
restricted during renegotiation] in RSSO server log
Workaround: Add the java proprerties for the JVM that is used by Tomcat.
"-Djdk.tls.allowUnsafeServerCertChange=true - Dsun.security.ssl.allowUnsafeRenegotiation=true"
Create \ Edit file <Tomcat>/bin/setenv.sh
JAVA_OPTS="-Djdk.tls.allowUnsafeServerCertChange=true" JAVA_OPTS="$JAVA_OPTS - D sun.security.ssl.allowUnsafeRenegotiation=true" export JAVA_OPTS
- Microsoft Windows
- Use tomcatXw.exe GUI.
- Create similar setenv.bat file with similar content.