Monitor types and attributes

The following monitor types/application classes are available in BMC PATROL for Hadoop.

Note

The JobTracker and TaskTracker are two essential processes involved in MapReduce execution in MRv1 (or Hadoop version 1). Both the processes are now deprecated in MRv2 (or Hadoop version 2) and replaced by Resource Manager, Application Master and Node Manager Daemons.

Note

The monitor types in Central Monitoring Administration are known as application classes in the PATROL Consoles.

The attributes in Central Monitoring Administration are known as parameters in the PATROL consoles.

You can access Central Monitoring Administration from the TrueSight console or ProactiveNet.

Top-level containers and configuration

DataNode monitoring

At the DataNode level, you can use the following application classes to monitor all DataNode related activities, including DataNode file system state, remote procedure activities, Java Virtual Machines metrics, user group information metrics, and other DataNode activities. 

JobHistory monitoring

The JobHistory server saves the history of all the running jobs in the Hadoop system.  The following application classes enable you to monitor the JobHistory service: 

JobTracker monitoring

The JobTracker is the service within Hadoop that farms out MapReduce tasks to specific nodes in the cluster, ideally the nodes that have the data, or at least are in the same rack. The JobTracker is a point of failure for the Hadoop MapReduce service. If it goes down, all running jobs are halted. The following application classes enable you to monitor the JobTracker service: 

NodeManager monitoring

A NodeManager accepts instructions from the ResourceManager and manage resources available on a single node. Use the following application classes to monitor the NodeManager:

NameNode Monitoring

The NameNode is the centerpiece of an HDFS file system. It keeps the directory tree of all files in the file system, and tracks where across the cluster the file data is kept. It does not store the data of these files itself. Client applications talk to the NameNode whenever they wish to locate a file, or when they want to add/copy/move/delete a file. The NameNode responds the successful requests by returning a list of relevant DataNode servers where the data lives. The following application classes enable you to monitor the NameNode: 

ResourceManagers monitoring

A ResourceManager is the master that arbitrates all the available cluster resources and thus helps manage the distributed applications running on the YARN system. It works together with the per-node NodeManagers and the per-application ApplicationMasters. Use the following application classes to monitor the ResourceManagers: 

Secondary NameNode monitoring

When the NameNode goes down and the file system goes offline, there is an optional SecondaryNameNode that can be hosted on a separate machine. It only creates checkpoints of the namespace by merging the edits file into the fsimage file and does not provide any real redundancy. Use the following application classes to monitor the secondary NameNode: 

TaskTracker monitoring

A TaskTracker is a node in the cluster that accepts tasks - Map, Reduce and Shuffle operations - from a JobTracker. Use the following application classes to monitor the TaskTracker node: 

JounalNodes monitoring

In a typical HA cluster, two separate machines are configured as NameNodes. At any point in time, exactly one of the NameNodes is in an Active state, and the other is in a Standby state. The Active NameNode is responsible for all client operations in the cluster, while the Standby is simply acting as a slave, maintaining enough state to provide a fast failover if necessary.

In order for the Standby node to keep its state synchronized with the Active node, both nodes communicate with a group of separate daemons called "JournalNodes".

Was this page helpful? Yes No Submitting... Thank you

Comments