Troubleshooting authentication issues
This topic provides information for troubleshooting authentication issues of Remedy Single Sign-On.
Generic authentication issues
Unable to define authentication chain for the client
When a user request is redirected to the Remedy SSO login URL, the following message is displayed: "Unexpected error happened. Not possible to define realm". This happens when the administrator deletes the default realm '*', adds another realm and does not configure a domain of the new realm.
Workaround: Do one of the following:
- Add default realm back with '*' as realm ID.
- Add the application host name or FQDN in the realm domain if the realm id is not '*'.
Remedy AR authentication issues
Remedy SSO AREA plugin error after Remedy SSO
The user completes Remedy SSO login (either on the Remedy SSO login page for AR authentication or on the IdP login page) and is redirected back to BMC application URL. Then, the application may display an authentication error. The AR AREA plugin log file (<AR>/Arserver/Db/arjavaplugin.log) might contain the following or similar error logs.
2015-09-13 17:04:21,324 ERROR [pool-4-thread-10] com.bmc.arsys.pluginsvr.plugins.ARPluginContext (?:?) - <ARSYS.AREA.RSSO>Could not validate userId with Service Provider. Could not retrieve user from authentication string. 2015-09-13 17:04:21,324 ERROR [pool-4-thread-10] com.bmc.arsys.pluginsvr.plugins.ARPluginContext (?:?) - <ARSYS.AREA.RSSO>Return Code:2
Workaround: To troubleshoot this error, perform the following steps.
- Check the file contents of '<AR>/Conf/rsso.cfg'. Ensure the value of 'sso-service-url' property is the Remedy SSO server URL, and it is accessible from the AR server node.
- You can use curl to test network connectivity from AR server node to Remedy SSO server URL.
- If connection test is OK, use curl go to the next step.
- Check if any proxy is used on AR Plug-in server for outgoing network connection.
- The proxy option must be defined in the Java options in armonitor.cfg: the startup string for the java plugin server would have Java params for enabling proxy.
- If proxy server is defined in Java plugin Server, force bypass of proxy for Remedy SSO server host name.
No groups for authenticated users
When AR 8.x is integrated with Remedy SSO and if authenticated users have no groups after login, uncomment the following setting in the <AR>/Conf/rsso.cfg file:
SAML authentication issues
IdP error on SAML request if SAML IdP login URL contains a query parameter
If the IdP login URL contains a query parameter, for example if you see '?' in the URLyou might be displayed an error when the browser is redirected to IdP login URL. Hence as a user you cannot continue the login process.
SAML IdP returns NameID with encrypted string
Some IdPs might return an encrypted string in the NameID of the response.
Workaround: If the NameID in a SAML response returned by IdP exceeds 255 characters, increase the size of the NameId column in the IssuesTokens table of the Remedy SSO
database. For example, if the NameID length is 300 characters, set the NameId column to at least VARCHAR(300).
Kerberos authentication issues
- Invalid keytab index number for Kerberos
- Browser sending NTLM instead of Kerberos
- Keytab file does not contain an entry to decrypt a service ticket
- Debugging Kerberos authentication issues
- Kerberos related events and messages
- Error: Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96 (or AP REP - AES128 CTS mode with HMAC SHA1-96)
- Increasing HTTP header size in Tomcat
Invalid keytab index number for Kerberos
This exception failure is generated in the logs when the keytab file is generated with a key version number (KVNO) number different from the one specified in the ticket. The solution is to regenerate the keytab file. Be sure to specify the /kvno 0 option; this ensures that the KVNO value is compatible.
amJAAS:10/18/2011 09:35:00:435 AM PDT: Thread[http-8443-1,5,main] Exception: com.sun.identity.authentication.spi.AuthLoginException: Failed to authentication. Failure unspecified at GSS-API level (Mechanism level: Specified version of key is not available (44))
Browser sending NTLM instead of Kerberos
Ensure that Remedy SSO server host name or domain is added in the list of websites for Kerberos authentication.
The entry "Authentication token is NTLM but not SPNEGO" in the log file indicates that the token that Remedy SSO sever receives from the client is a Microsoft Windows NT LAN Manager (NTLM) token and not a Kerberos token as required. The failure could be caused for the following reasons:
- Browser is not correctly configured to use Kerberos authentication. Refer Configuring the browser.
- Problem related to obtaining a service token for the Remedy SSO server. Make sure that Kerberos Service ticket is obtained when it tries to get an access to Remedy SSO server on client's machine, it should look as something like the below where
is the hostname of the machine with Remedy SSO server (this check guarantees that customer's machine asked it from KDC and it then issued the service ticket).
c:\Windows\system32\klist.exe ... #1> Client: dev_adei @ QA2R.LOCAL Server: HTTP/access.bmc.com @ QA2R.LOCAL KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96 Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize Start Time: 11/3/2016 9:00:43 (local) End Time: 11/3/2016 19:00:43 (local) Renew Time: 11/10/2016 9:00:43 (local) Session Key Type: AES-256-CTS-HMAC-SHA1-96 ...
Keytab file does not contain an entry to decrypt a service ticket
If you get the following events, it indicates that the keytab file does not contain an entry to decrypt a service ticket.
java.security.PrivilegedActionException: GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed) Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed) Caused by: KrbException: Checksum failed Caused by: java.security.GeneralSecurityException: Checksum failed
Examine items and make sure that SPNs are valid. In case if SPN password is used in the Remedy SSO Admin Console, ensure that the "Service Principal Name" is specified as
To see content of the existing keytab file, run the ktab utility as follows:
c:\java sun.security.krb5.internal.tools.Ktab -l -e -t -k all.keytab Keytab name: all.keytab KVNO Timestamp Principal ---- ----------------- ------------------------------------------------------------------------------------ 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (1:DES CBC mode with CRC-32) 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (3:DES CBC mode with MD5) 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (23:RC4 with HMAC) 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (18:AES256 CTS mode with HMAC SHA1-96) 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (17:AES128 CTS mode with HMAC SHA1-96)
Note, that this keytab file contains five entries for the same principal, but each entry has a different encryption type. You must use the option "/crypto all" with the ktpass utility to generate the keytab file.
Debugging Kerberos authentication issues
You can perform the following tasks while debugging Kerberos authentication:
Run the following command to ensure that customer's machine has joined the domain and a domain user is used:
- Ensure that the customer has other internal resources with Kerberos authentication and the customer can successfully log into them and use them from his machine. For this, you must have service tickets for them in the output of the klist utility.
- Ensure that Remedy SSO server is configured to use the same domain that the customer machine has joined.
- Ensure that that you are trying to access the Remedy SSO server using its FQDN, for example, http://access.example.com/rsso/. Also, the host name used in the FQDN is identical to the host name used in SPN for a service account created in Key Distribution Center. In this case, the SPN will be HTTP/access.example.com.
- Ensure that the Kerberos TGT ticket is obtained on the customer machine.
- Ensure that the browser is configured properly according to steps described in Configuring Kerberos authentication.
- Ensure that the KDC domain is defined in uppercase in the Remedy SSO Admin Console.
- Ensure that the time difference between KDC and the customer machine is no more than 5 minutes.
- Ensure that the Kerberos Service ticket obtained on the customer machine accessing the Remedy SSO server looks like HTTP/access.bmc.com@RSSO.COM where where access.bmc.com@RSSO.COM is the host name of the machine that hosts the Remedy SSO server.
Kerberos related events and messages
Ensure that the Remedy SSO logging level is set to TRACE.You can find the Kerberos related events and log information in the following files that are normally located in the log directory of Tomcat.
- rsso.log: Main log file for Remedy SSO server.
- tomcat8-stdout.*.log: Kerberos related events from Java Authentication and Authorization Service that is used by Remedy SSO server internally to authenticate users.
Error: Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96 (or AP REP - AES128 CTS mode with HMAC SHA1-96)
If you find this error in the rsso.log file, you have to install JCE Unlimited Strength Jurisdiction Policy Files for JDK/JRE to support AES128 and AES256 encryption types. You can find the policy files at the following links:
- Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files for JRE 8
- Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files for JRE 7
Also, you need to install these policy files in JRE that is used by the Tomcat on which the Remedy SSO runs.
Increasing HTTP header size in Tomcat
The Kerberos service ticket is passed as a header value in the http request. Though the default maximum header size in tomcat is 4096 (4KB), under some circumstances the header size may go up to 28 KB. The login fails because of the large size, and the browser displays an error message because Tomcat does not respond to such requests.
To fix this issue, specify a maxHttpHeaderSize attribute on the https connector and set a large enough value in bytes.
LDAP authentication issues
LDAP authentication failure
|LDAP authentication failure|
When using LDAP (LDAP over SSL) as the authentication method in an environment that uses Java 7.75+ / Java 8.31+, random users are not authenticated with the following exceptions:
Add the java proprerties for the JVM that is used by Tomcat.
Create or edit the file <Tomcat>/bin/setenv.sh
Microsoft Windows example:
|The login request is redirected to an empty rsso/start URL|
When your your Remedy SSO server and the integrated application both use invalid TLS/SSL certificates for HTTPS connection, the certificate confirmation dialog breaks the flow, and you cannot login by using Microsoft Edge and Safari browsers.
|Use another browser to log in, or open the application URL again after confirming the exception for the certificate.|
Authentication issues for applications hosted on different domains
Unable to issue id_token
After configuring the Remedy SSO agent for using the "openid" scope, the Remedy SSO server fails to generate id_token in the following scenarios and an exception is logged in the Remedy SSO agent logs:
For more information about the exception, see the Remedy SSO agent logs.
After configuring the Remedy SSO agent for using the "openid" scope, ensure the following:
An OAuth2 client is not able to use the OpenID scope
At the time of registering a client as an OAuth2 client, if you do not select the openid (Scope used for OpenID connect) check box, the client is not able to use the openid scope. The Remedy SSO server logs a message mentioning that the specified OAuth2 client is not allowed to use the openid scope.
|On the Admin Console, edit the OAuth2 client details and select the openid (Scope used for OpenID connect) check box for that client.|
|Id_token is invalid|
The user is unable to log and gets the following error message:
Synchronize the time on the Remedy SSO server and Remedy SSO agent machines.