End-to-end steps for configuring Active Directory Kerberos authentication

The following topics are provided:

Overview

The Microsoft Windows Server operating system implements the Kerberos version 5 authentication protocol. Windows Server operating system also implements extensions for public key authentication. The Kerberos authentication client is implemented as a security support provider (SSP) and can be accessed through the Security Support Provider Interface (SSPI). Initial user authentication is integrated with the Winlogon single sign-on architecture. The Kerberos Key Distribution Center (KDC) is integrated with other Windows Server security services running on the domain controller. The KDC uses the domain’s Active Directory service database as its account database. An Active Directory server is required for default Kerberos implementations.

Before you begin

To make changes to Microsoft Windows Active Directory, you must have administrator permissions on the domain controller computer and in the domain itself. To make changes to the BMC Atrium Single Sign-On server, you must also have administrator permissions for the BMC Atrium SSO Admin Console.

Configuring Kerberos authentication with Active Directory

Recommendations

If you are using Microsoft Windows Active Directory version 2003, you might need to update to a later version to get setspn options. For more information, see http://support.microsoft.com/kb/970536.

Perform the following tasks to configure Kerberos with Active Directory.

No.	Task	Description
1	Understanding how Kerberos works	Kerberos is a network authentication protocol. It is designed to provide strong authentication for client/server applications by using secret-key cryptography. To understand the conceptual framework, see Kerberos-authentication.
2	Creating user identity which will be used for active directory authentication	You must log on to the domain controller computer as a user with administrator permissions. Then, create a user in Active Directory server for authentication. Enter the user's First name and User logon name. Specify the Password and confirm the password. Select the User cannot change password and Password never expires check boxes. Verify that you have not selected the Require preauthentication check box.
Note: BMC Atrium Single Sign-On server communicates with your KDC or Active Directory domain controller on TCP/UDP Port 88. Refer to the Microsoft documentation for specific information related to your Active Directory version, http://technet.microsoft.com/en-us/library/dd772723%28v=ws.10%29.aspx.
3	*(Optional)* Generate a keytab file for the service principal	After the accounts for the service principals are created, a keytab file must be generated. For more information, see Generating-a-keytab-file-for-the-service-principal. Alternatively, you can add an Service Principal Name (SPN) password. (If you are using an SPN password, you must map the SPN to a user account, and the user account must have an ID that matches the SPN.)
4	Mapping the Kerberos service name	Add an SPN for mapping the Kerberos service name. The setspn.exe utility allows manipulation of SPNs within Active Directory. For more information, see Mapping-the-Kerberos-service-name.
5	Configuring the Kerberos module	After you have generated a keytab file and mapped the Kerberos service name, configure the Kerberos module on the BMC Atrium SSO Admin Console. For more information, see Configuring-the-Kerberos-module.
6	Reconfiguring your browser	If you have not reconfigured your browser for using Kerberos authentication, you must configure it. For more information, see Reconfiguring-your-browser.
7	*(Optional)* Chaining different modules	If a complex authentication chain is needed, you can create a certificate chain by using the Realm Editor on the BMC Atrium SSO Admin Console. Perform the procedures in Chaining-different-modules.
8	Testing Kerberos authentication	Make sure that the test machine belongs to the KDC domain and the user's details are in the Active Directory account database. Then, enter the application URL in the browser. The user must be authenticated automatically. Authentication chaining (including Kerberos authentication) can be tested without binding to the particular agent. Open the Login URL in a new browser instance. In case of successful Kerberos authentication, the following message is displayed to the user without being prompted for credentials. "You (<username>) are logged in with insufficent privileges for the admin console."

Note

In High Availability (HA) mode, when you create the keytab file and the SPN mapping, use the name of the load balancer host instead of the name of the BMC Atrium Single Sign-On server host.

Troubleshooting Kerberos authentication

If you encounter issues related to Kerberos authentication, refer to the Kerberos troubleshooting section. For more information, see Troubleshooting-Kerberos-authentication.