EMR Cluster (AWS_EMR_CLUSTER)
Attributes (parameters)
The following attributes are available for this monitor type:
Name | Description | Unit | Default Performance Key Indicator (KPI) |
---|---|---|---|
Apps Completed (AppsCompleted) | Displays the number of applications submitted to YARN that are completed. | # | No |
Apps Failed (AppsFailed) | Displays the number of applications submitted to YARN that are killed. | # | No |
Apps Killed (AppsKilled) | Displays the number of applications submitted to YARN that are killed. | # | No |
Apps Pending (AppsPending) | Displays the number of applications submitted to YARN that are in pending state. | # | No |
Apps Running (AppsRunning) | Displays the number of applications submitted to YARN that are running. | # | No |
Apps Submitted (AppsSubmitted) | Displays the number of applications submitted to YARN. | # | No |
Capacity Remaining (CapacityRemaining) | Displays the amount of remaining HDFS disk capacity. | GB | No |
Container Allocated (ContainerAllocated) | Displays the number of resource containers allocated by the ResourceManager. | # | No |
Container Pending (ContainerPending) | Displays the number of containers in the queue that are not yet allocated. | # | No |
Container Pending Ratio (ContainerPendingRatio) | Displays the ratio of pending containers to containers allocated (ContainerPendingRatio = ContainerPending / ContainerAllocated). If ContainerAllocated = 0, then ContainerPendingRatio = ContainerPending. The value is useful for scaling cluster resources based on container allocation behavior. | # | No |
Container Reserved (ContainerReserved) | Displays the number of containers that are reserved. | # | No |
Core Nodes Pending (CoreNodesPending) | Displays the number of core nodes waiting to be assigned. All of the core nodes requested may not be immediately available; this metric reports the pending requests. Data points for this metric are reported only when a corresponding instance group exists. | # | No |
Core Nodes Running (CoreNodesRunning) | Displays the number of core nodes that are working. Data points for this metric are reported only when a corresponding instance group exists. | # | Yes |
Corrupt Blocks (CorruptBlocks) | Displays the number of blocks that are reported as corrupt by HDFS. | # | No |
Dfs Pending Replication Blocks (DfsPendingReplicationBlocks) | Displays the status of block replication: blocks being replicated, the age of replication requests, and unsuccessful replication requests. | # | No |
Hbase Backup Failed (HbaseBackupFailed) | Displays whether the last backup failed. This is set to 0 by default and updated to 1 if the previous backup attempt failed. This metric is only reported for HBase clusters. | 0 - OK, 1 - Failed | No |
HDFS Bytes Read (HDFSBytesRead) | Displays the number of bytes read from HDFS. | # | No |
HDFS Bytes Written (HDFSBytesWritten) | Displays the number of bytes written to HDFS. | # | No |
HDFS Utilization (HDFSUtilization) | Displays the percentage of HDFS storage currently used. | % | No |
Is Idle (IsIdle) | Displays the number of resource containers allocated by the ResourceManager. | 0 - Not Idle, 1 - Idle | Yes |
Live Data Nodes (LiveDataNodes) | Displays the percentage of data nodes that are receiving work from Hadoop. | % | No |
Memory Allocated (MemoryAllocated) | Displays the amount of memory allocated to the cluster. | MB | No |
Memory Available (MemoryAvailable) | Displays the amount of memory available for allocating. | MB | No |
Memory Reserved (MemoryReserved) | Displays the amount of memory reserved. | MB | No |
Memory Total (MemoryTotal) | Displays the total amount of memory in the cluster. | MB | No |
Missing Blocks (MissingBlocks) | Displays the number of blocks in which HDFS has no replicas. These might be corrupt blocks. | # | No |
Most Recent Backup Duration (MostRecentBackupDuration) | Displays the amount of time taken to complete the most recent backup. This metric is set regardless of whether the last backup succeeded or failed. While the backup is ongoing, this metric returns the number of minutes after the backup started. This metric is only reported for HBase clusters. | minutes | No |
MR Active Nodes (MRActiveNodes) | Displays the number of nodes presently running MapReduce tasks or jobs. | # | No |
MR Decommissioned Nodes (MRDecommissionedNodes) | Displays the number of nodes allocated to MapReduce applications that have been marked in a DECOMMISSIONED state. | # | No |
MR Lost Nodes (MRLostNodes) | Displays the number of nodes allocated to MapReduce that have been marked in a LOST state. | # | No |
MR Rebooted Nodes (MRRebootedNodes) | Displays the number of rebooted nodes available for MapReduce. | # | No |
MR Total Nodes (MRTotalNodes) | Displays the number of nodes presently available to MapReduce jobs. | # | No |
MR Unhealthy Nodes (MRUnhealthyNodes) | Displays the number of nodes available to MapReduce jobs marked in an UNHEALTHY state. | # | No |
Pending Deletion Blocks (PendingDeletionBlocks) | Displays the number of blocks marked for deletion. | # | No |
S3 Bytes Read (S3BytesRead) | Displays the number of bytes read from Amazon S3. | # | No |
S3 Bytes Written (S3BytesWritten) | Displays the number of bytes written to Amazon S3. | # | No |
Time Since Last Successful Backup (TimeSinceLastSuccessfulBackup) | Displays the number of elapsed minutes after the last successful HBase backup started on your cluster. This metric is only reported for HBase clusters. | minutes | No |
Total Load (TotalLoad) | Displays the total number of concurrent data transfers. | # | No |
Under Replicated Blocks (UnderReplicatedBlocks) | Displays the number of blocks that need to be replicated. | # | No |
YARN Memory Available Percentage (YARNMemoryAvailablePercentage) | Displays the percentage of remaining memory available to YARN (YARNMemoryAvailablePercentage = MemoryAvailableMB / MemoryTotalMB). This value is useful for scaling cluster resources based on YARN memory usage. | % | No |