Configuring the Hadoop monitor type


This topic provides information about configuring PATROL for Hadoop KM in the Central Monitoring Administration and TrueSight console. To configure the KM, set the following preferences on the Add Monitoring Configuration dialog.


Hadoop configuration video

Click to view a short video on how to configure Hadoop for monitoring with Kerberos authentication.

icon-play.png https://youtu.be/_pDQUpHlru4

Configuration details

Monitoring Solution

Version

Monitor Profile

Monitor Type

Hadoop

1.1.00

Hadoop

Hadoop

Click Add button.png button to add the Hadoop Configuration details. 

Field

Description

Environment Configuration

Environment name

Enter a logical unique name for the environment. A container will be created with the environment name. This environment will hold all the hosts. BMC recommends you to provide only alphanumeric characters in environment name.

Valid values: a-z, A-Z, 0-9

Invalid values: # ' . | ? \ " [ ] + = &

Hadoop connection details

Hadoop Vendor

Select the monitored Hadoop vendor. The supported vendors are:

  • Apache Hadoop
  • IBM BigInsights
  • MapR
  • Cloudera
  • Horton

Hadoop Host

Enter a unique, user-defined hostname or an IP address.

Hadoop Port

Enter the port number of the hadoop component that you want to monitor. Hadoop component can be any of the following:

NameNode, ResourceManager, DataNode, NodeManager, JobHistory, JournalNode, JobTracker, and TaskTracker.

Default port number of NameNode application class is 50070.

Hadoop Authentication

Select the authentication from the following:

  • None
  • Basic
  • Kerberos

Hadoop User

Enter the username to connect to the Hadoop server. Depending on the Hadoop environment, the Hadoop username may not be required.

Hadoop Password

Enter the password (if required) to connect to the Hadoop server.

Kerberos authentication details

krb5 file path

Enter the Kerberos file path.

Note

  • The following are the steps required to monitor Hadoop with kerberos authentication:


      1. Select Kerberos under Hadoop Authentication section.
      2. The Kerberos username and password fields are required.
      3. Enter the full path of your hadoop krb5.conf file.

The krb5.conf file contains Kerberos configuration information. This includes information describing the default Kerberos realm, and the location of the Kerberos key distribution centers for known realms, and mappings of hostnames onto Kerberos realms. 

HTTPs secure details

Use HTTPs

Select this checkbox if you want to use HTTPs protocol for connection. By default, HTTP protocol is used for the connection.

TrustStore file (jks)

If you select to use HTTPs protocol for connection, you must provide the location to the Java Keystore File (JKS) for a successful connection to the URL in a secure mode.

TrustStore password

Enter the password to the JKS file.

Java collector settings

JAVA_HOME

Enter the directory where you have installed Java version 1.7 or later.

JVM Argments

Enter the JVM arguments for the Hadoop java collector.

Executing user

Enter the username that has permissions to run Java, only if the monitoring user does not have permissions.

Executing password

Enter the executing user password.

Local admin OS Credentials

The local user with permissions to run the Java. 

User Name

Enter the local username.

Password

Enter the local user password.

Confirm Password

Re-enter the password for confirmation.

Collection details

Collection level

Select the collection level for monitoring the Hadoop instances.

  1. Availability Enables monitoring of the Hadoop instance availability (if instance is running or is down) of one of the following instances existing in the environment
    1. NameNode
    2. SecondaryNameNode
    3. ResourceManager
    4. DataNode
    5. NodeManager
    6. JobHistory
    7. JobTracker
    8. TaskTracker
    9. JournalNode
  2. Full data monitoring - Enables the monitoring of all Hadoop instances.
  3. Custom data monitoring - Manually select the Hadoop instances to monitor.

Custom Hadoop Metiric

Select the list of metrics for monitoring.

  1. NameNode Metrics
    • NameNode All - Select this checkbox to monitor all NameNode metrics
    • Selection - Select the NameNode metrics you want to monitor
  2. DataNode Metrics
    • DataNode All - Select this checkbox to monitor all DataNode metrics
    • Selection - Select the DataNode metrics you want to monitor
  3. ResourceManager Metrics
    • ResourceManager All - Select this checkbox to monitor all ResourceManager metrics
    • Selection - Select the ResourceManager metrics you want to monitor
  4. JobTracker Metrics
    • JobTracker All - Select this checkbox to monitor all JobTracker metrics
    • Selection - Select the JobTracker metrics you want to monitor
  5. NodeManager Metrics
    • NodeManager All - Select this checkbox to monitor all NodeManager metrics
    • Selection - Select the NodeManager metrics you want to monitor
  6. TaskTracker Metrics
    • TaskTracker All - Select this checkbox to monitor all TaskTracker metrics
    • Selection - Select the TaskTracker metrics you want to monitor
  7. SecondaryNameNode Metrics
    • SecondaryNameNode All - Select this checkbox to monitor all SecondaryNameNode metrics
    • Selection - Select the SecondaryNameNode metrics you want to monitor
  8. JobHistory Server Metrics
    • JobHistoryServer All - Select this checkbox to monitor all JobHistory Server metrics
    • Selection - Select the JobHistory Server metrics you want to monitor
  9. JournalNode Metrics
    • JournalNode All - Select this checkbox to monitor all JournalNode metrics
    • Selection - Select the JournalNode metrics you want to monitor

Filtering 

DataNode filtering options

Filtering mode

Specify the filtering type for filtering the DataNodes

  • None - Selects all available DataNodes for filtering
  • Include - Includes only the DataNodes entered in the Include/Exclude from Monitoring field
  • Exclude - Excludes the DataNodes entered in the Include/Exclude from Monitoring field

Include/Exclude from Monitoring (regex)

Specify a regular expression or the DataNode names to either include or exclude DataNodes from filtering. Use comma (,) only to separate the DataNodes.

NodeManager filtering options

Filtering mode

Specify the filtering type for filtering the DataNodes

  • None - Selects all available NodeManagers for filtering
  • Include - Includes only the NodeManagers entered in the Include/Exclude from Monitoring field
  • Exclude - Excludes the NodeManagers entered in the Include/Exclude from Monitoring field

Include/Exclude from Monitoring (regex)

Specify a regular expression or the NodeManager names to either include or exclude NodeManagers from filtering. Use comma (,) only to separate the NodeManagers.

KM Administration

Logging

Select this option to enable PSL and Java logging.

Availability collection time (min)

Select the availability collection time in minutes.

Data collection time (min)

Select the metrics collection time in minutes.

Device mapping

Select to enable device mapping to consolidate the configured devices at the device level.

Add to List

Click this option to add the configuration to the List - Environment Configurations list.

Modify Selection

Click this option to modify the configuration details.

Remove from List

Click this option to remove the configuration from the List - Environment Configurations list.

List - Environment Configurations

Displays a list of environments you have configured.

Reset

Click this option to reset the monitor configuration to its default values.

Add

Click this option to add the configuration.

Close

Click this option to close the Add Monitor Types configuration screen.


 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*