Troubleshooting authentication issues

Remedy SSO AREA plugin error after Remedy SSO

The user completes Remedy SSO login (either on the Remedy SSO login page for AR authentication or on the IdP login page) and is redirected back to BMC application URL. Then, the application may display an authentication error. The AR AREA plugin log file (<AR>/Arserver/Db/arjavaplugin.log) might contain the following or similar error logs.

2015-09-13 17:04:21,324 ERROR [pool-4-thread-10] com.bmc.arsys.pluginsvr.plugins.ARPluginContext (?:?) - <ARSYS.AREA.RSSO>Could not validate userId with Service Provider. Could not retrieve user from authentication string.

2015-09-13 17:04:21,324 ERROR [pool-4-thread-10] com.bmc.arsys.pluginsvr.plugins.ARPluginContext (?:?) - <ARSYS.AREA.RSSO>Return Code:2

Workaround: To troubleshoot this error, perform the following steps.

1. Check the file contents of '<AR>/Conf/rsso.cfg'. Ensure the value of 'sso-service-url' property is the Remedy SSO server URL, and it is accessible from the AR server node.
   • You can use curl to test network connectivity from AR server node to Remedy SSO server URL.
2. If connection test is OK, use curl go to the next step.
3. Check if any proxy is used on AR Plug-in server for outgoing network connection.
   • The proxy option must be defined in the Java options in armonitor.cfg: the startup string for the java plugin server would have Java params for enabling proxy.
   • If proxy server is defined in Java plugin Server, force bypass of proxy for Remedy SSO server host name.

No groups for authenticated users

When AR 8.x is integrated with Remedy SSO and if authenticated users have no groups after login, uncomment the following setting in the  <AR>/Conf/rsso.cfg file:

AR-USER-GROUPS-FIX: true

IdP error on SAML request if SAML IdP login URL contains a query parameter

If the IdP login URL contains a query parameter, for example if you see '?' in the URL https://sso.connect.pingidentity.com/sso/idp/SSO.saml2?idpid=XXX, you might be displayed an error when the browser is redirected to IdP login URL. Hence as a user you cannot continue the login process.

SAML IdP returns NameID with encrypted string

Some IdPs might return an encrypted string in the NameID of the response.

Workaround: If the NameID in a SAML response returned by IdP exceeds 255 characters, increase the size of the NameId column in the IssuesTokens table of the Remedy SSO

database. For example, if the NameID length is 300 characters, set the NameId column to at least VARCHAR(300).

Invalid keytab index number for Kerberos

This exception failure is generated in the logs when the keytab file is generated with a key version number (KVNO) number different from the one specified in the ticket. The solution is to regenerate the keytab file. Be sure to specify the /kvno 0 option; this ensures that the KVNO value is compatible.

amJAAS:10/18/2011 09:35:00:435 AM PDT: Thread[http-8443-1,5,main]
Exception: com.sun.identity.authentication.spi.AuthLoginException: Failed to authentication.
Failure unspecified at GSS-API level (Mechanism level: Specified version of key is not available (44))

Browser sending NTLM instead of Kerberos

Ensure that Remedy SSO server host name or domain is added in the list of websites for Kerberos authentication.

The entry "Authentication token is NTLM but not SPNEGO" in the log file indicates that the token that Remedy SSO sever receives from the client is a Microsoft Windows NT LAN Manager (NTLM) token and not a Kerberos token as required. The failure could be caused for the following reasons:

• Browser is not correctly configured to use Kerberos authentication. Refer Configuring the browser.

• Problem related to obtaining a service token for the Remedy SSO server. Make sure that Kerberos Service ticket is obtained when it tries to get an access to Remedy SSO server on client's machine, it should look as something like the below where access.bmc.com  is the hostname of the machine with Remedy SSO server (this check guarantees that customer's machine asked it from KDC and it then issued the service ticket).

c:\Windows\system32\klist.exe
...
#1> Client: dev_adei @ QA2R.LOCAL
    Server: HTTP/access.bmc.com @ QA2R.LOCAL
    KerbTicket Encryption Type: AES-256-CTS-HMAC-SHA1-96
    Ticket Flags 0x40a10000 -> forwardable renewable pre_authent name_canonicalize
    Start Time: 11/3/2016 9:00:43 (local)
    End Time:   11/3/2016 19:00:43 (local)
    Renew Time: 11/10/2016 9:00:43 (local)
    Session Key Type: AES-256-CTS-HMAC-SHA1-96
...

Keytab file does not contain an entry to decrypt a service ticket

If you get the following events, it indicates that the keytab file does not contain an entry to decrypt a service ticket.

java.security.PrivilegedActionException: GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)
Caused by: GSSException: Failure unspecified at GSS-API level (Mechanism level: Checksum failed)
Caused by: KrbException: Checksum failed
Caused by: java.security.GeneralSecurityException: Checksum failed

Examine items and make sure that SPNs are valid. In case if SPN password is used in the Remedy SSO Admin Console, ensure that the "Service Principal Name" is specified as HTTP/access.bmc.com@RSSO.COM.

To see content of the existing keytab file, run the  ktab utility as follows:

c:\java sun.security.krb5.internal.tools.Ktab -l -e -t -k all.keytab
Keytab name: all.keytab
KVNO Timestamp Principal
---- ----------------- ------------------------------------------------------------------------------------
 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (1:DES CBC mode with CRC-32)
 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (3:DES CBC mode with MD5)
 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (23:RC4 with HMAC)
 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (18:AES256 CTS mode with HMAC SHA1-96)
 0 12/31/69 12:00 PM HTTP/access.bmc.com@RSSO.COM (17:AES128 CTS mode with HMAC SHA1-96)

Note, that this keytab file contains five entries for the same principal, but each entry has a different encryption type. You must use the option "/crypto all"  with the ktpass utility to generate the keytab file.

Debugging Kerberos authentication issues

You can perform the following tasks while debugging Kerberos authentication:

• Run the following command to ensure that customer's machine has joined the domain and a domain user is used:

  C:\whoami
  DOMAIN\userID

• Ensure that the customer has other internal resources with Kerberos authentication and the customer can successfully log into them and use them from his machine. For this, you must have service tickets for them in the output of the klist utility.
• Ensure that Remedy SSO server is configured to use the same domain that the customer machine has joined.
• Ensure that that you are trying to access the Remedy SSO server using its FQDN, for example, http://access.example.com/rsso/. Also, the host name used in the FQDN is identical to the host name used in SPN for a service account created in Key Distribution Center. In this case, the SPN will be HTTP/access.example.com.
• Ensure that the Kerberos TGT ticket is obtained on the customer machine.
• Ensure that the browser is configured properly according to steps described in Configuring Kerberos authentication.
• Ensure that the KDC domain is defined in uppercase in the Remedy SSO Admin Console.
• Ensure that the time difference between KDC and the customer machine is no more than 5 minutes.
• Ensure that the Kerberos Service ticket obtained on the customer machine accessing the Remedy SSO server looks like HTTP/access.bmc.com@RSSO.COM where where access.bmc.com@RSSO.COM is the host name of the machine that hosts the Remedy SSO server.

Kerberos related events and messages

Ensure that the Remedy SSO logging level is set to TRACE.You can find the Kerberos related events and log information in the following files that are normally located in the log directory of Tomcat.

• rsso.log: Main log file for Remedy SSO server.
• tomcat8-stdout.*.log: Kerberos related events from Java Authentication and Authorization Service that is used by Remedy SSO server internally to authenticate users.

Error: Cannot find key of appropriate type to decrypt AP REP - AES256 CTS mode with HMAC SHA1-96 (or AP REP - AES128 CTS mode with HMAC SHA1-96)

If you find this error in the rsso.log file, you have to install JCE Unlimited Strength Jurisdiction Policy Files for JDK/JRE to support AES128 and AES256 encryption types. You can find the policy files at the following links:

• Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files for JRE 8
• Java Cryptography Extension (JCE) Unlimited Strength Jurisdiction Policy Files for JRE 7

Also, you need to install these policy files in JRE that is used by the Tomcat on which the Remedy SSO runs.

Increasing HTTP header size in Tomcat

The Kerberos service ticket is passed as a header value in the http request. Though the default maximum header size in tomcat is 4096 (4KB), under some circumstances the header size may go up to 28 KB. The login fails because of the large size, and the browser displays an error message because Tomcat does not respond to such requests.

To fix this issue, specify a maxHttpHeaderSize attribute on the https connector and set a large enough value in bytes.