This plugin
collects metrics from Apache Hadoop nodes.
Prerequisites
Meter 4.5 or later must be installed.
The Hadoop plugin supports the following Operating System.
Linux | ✅️ |
To install the plugin
- From the top right of the screen, perform one of the following actions:
- Click
> Data Collection, select the TrueSight meter, and select the Sources tab. - Click > Configure Sources.
- Click
- Use the search box or scroll through the page to find the source system which has the TrueSight meter on which you want to install the plugin.
- Click the name of the source to view source details.
- Select the Plugins tab.
- Use the search box or scroll through the page to find the plugin you want to install.
Click + Install to start the installation.
Refer to the following sections for the configuration details required to collect data and view the list of plugin metrics.
Configuration details
Field | Description |
|---|---|
Host | The hadoop host name |
Type | Type of hadoop node (single node or multi node) to monitor |
RM Port | Resource manager hadoop port number |
DataNode Port | DataNode hadoop port number |
NameNode Port | NameNode hadoop port number |
PollInterval | The frequency (in milliseconds) used to poll for metrics. Default value is 1000 ms, we do not recommend a value smaller than the default value |
Plugin metrics
Metric Name | Description |
|---|---|
HADOOP_NAMENODE_FS_CAPACITY_REMAINING | Remaining disk space left in bytes |
HADOOP_NAMENODE_FS_CORRUPTED_BLOCKS | Number of corrupt blocks |
HADOOP_NAMENODE_FS_MISSINGED_BLOCKS | Number of missing blocks |
HADOOP_NAMENODE_FS_BLOCKSED_TOTAL | Total number of blocks |
HADOOP_NAMENODE_FS_FILES_TOTAL | Total number of files |
HADOOP_NAMENODE_FS_BLOCKED_CAPACITY | Total Blocked Capacitys |
HADOOP_NAMENODE_FS_UNDER_REPLICATED_BLOCKS | Number of under replicated blocks |
HADOOP_NAMENODE_FS_CAPACITYED_USED | Disk usage in bytes |
HADOOP_NAMENODE_FS_TOTAL_LOAD | Total load on the file system |
HADOOP_NAMENODE_FSSTATE_VOLUME_FAILURES_TOTAL | Total volume failures |
HADOOP_NAMENODE_FSSTATE_NUMBER_OF_LIVE_DATA_NODES | Total number of live data nodes |
HADOOP_NAMENODE_FSSTATE_NUMBER_OF_DEAD_DATA_NODES | Total number of dead data nodes |
HADOOP_NAMENODE_FSSTATE_NUMBER_OF_STALE_DATA_NODES | Number of stale data nodes |
HADOOP_MAP_REDUCED_PROCESS_CPU_TIME | Prcesss cpu time |
HADOOP_MAP_REDUCED_PROCESS_CPU_LOAD | Prcesss cpu load |
HADOOP_MAP_REDUCED_SYSTEM_CPU_LOAD | System cpu load |
HADOOP_MAP_REDUCED_AVAILABLE_PROCESSORS | Number of processors available |
HADOOP_MAP_REDUCED_TOTAL_SWAP_SPACE_SIZE | Total swap space size |
HADOOP_MAP_REDUCED_FREE_SWAP_SPACE_SIZE | Total swap space free |
HADOOP_MAP_REDUCED_FREE_PHYSICAL_MEMORY_SIZE | Physical memory free size |
HADOOP_MAP_REDUCED_TOTAL_PHYSICAL_MEMORY_SIZE | Total Physical memory size |
HADOOP_YARN_APPLICATION_RUNNING | The number of running apps |
HADOOP_YARN_APPLICATION_FAILED | The number of failed apps |
HADOOP_YARN_APPLICATION_KILLED | The number of killed apps |
HADOOP_YARN_APPLICATION_PENDING | The number of pending apps |
HADOOP_YARN_AVAILABLE_MEMORY | The amount of available memory |
HADOOP_YARN_AVAILABLE_VCORES | The number of available virtual cores |
HADOOP_YARN_NUMMBER_OF_UNHEALTHY_NODES | The number of unhealthy nodes |
HADOOP_YARN_NUMMBER_OF_ACTIVE_NODES | The number of active nodes |
HADOOP_YARN_NUMMBER_OF_LOST_NODES | The number of lost nodes |
HADOOP_YARN_USED_MEMORY | Total used memory |
HADOOP_YARN_NUMBER_OF_CONTAINERS | Number of containers |
HADOOP_DATANODE_HEAP_MEMORY_USED | Heap memory used |
HADOOP_DATANODE_HEAP_MEMORY_MAX | Heap memory max used |
HADOOP_DATANODE_GC_COUNT | GC count |
HADOOP_DATANODE_GC_TIME_MILLIS | GC time milliseconds |
HADOOP_DATANODE_GC_NUMBER_OF_WARN_THREADSHOLD_EXCEEDED | GC of warn threadshold exceeded |
HADOOP_DATANODE_GC_NUMBER_OF_INFO_THREADSHOLD_EXCEEDED | GC of info threadshold exceeded |
HADOOP_DATANODE_GC_TOTAL_EXTRA_SLEEP_TIME | GC extra sleep time |
HADOOP_DATANODE_BLOCKS_READ | Blocks read |
HADOOP_DATANODE_BLOCKS_WRITTEN | Blocks written |
HADOOP_DATANODE_VOLUME_FAILURES | Number of failed volumes |
HADOOP_DATANODE_NUMBER_OF_FAILED_STORAGE_VOLUMES | Number of failed storage volumes |
HADOOP_DATANODE_REMAINING_CAPACITY | Remaining disk space |