Collecting additional metrics using the Stackdriver monitoring
The Google Cloud Platform - GCP API Extractor collects a default set of performance metrics from your Google Cloud Platform infrastructure. In addition to the default metrics, you can use the Monitoring agent to collect additional data such as metrics and logs from third-party applications. These metrics are useful for investigating the capacity-related issues that might occur in your Google Cloud environment. When you run the GCP API ETL, these metrics are imported into the database.
Before installing the Monitoring agent, you must create and enable a Stackdriver account. A Stackdriver account contains the monitoring and other configuration information about Google Cloud Platform projects. To collect Stackdriver metrics, your GCP projects must be associated with a Stackdriver account. You can associate projects while creating a Stackdriver account or can associate them later. You can create any number of Stackdriver accounts.
- Create a Stackdriver account.
- Log in to the Google Cloud Platform console with your Google account credentials.
- From the project list at the upper-left corner next to Google Cloud Platform, select the project that you want to enable for Stackdriver monitoring.
- In the left navigation pane, click Monitoring under Stackdriver.
- Select Create a new Stackdriver account, and click Continue.
- On the Create your free Stackdriver account page, verify the project name in the Google Cloud Platform project list, and click Create Account.
- (Optional) Select additional Google Cloud Platform projects that you want to enable for Stackdriver monitoring, and click Continue.
- (Optional) Add AWS accounts to monitor as a part of this Stackdriver account. Otherwise, click Skip AWS Setup.
- Read the instructions for installing Stackdriver agents, and click Continue.
- On the Get Reports by Email page, select a frequency of receiving the performance report by email, and click Continue.
- After you gather the required information, click Launch Monitoring.
The Stackdriver Monitoring page is displayed, which indicates that your Stackdriver account is ready for use.
- Install the Monitoring agent on each of your virtual machines.
The Monitoring agent is available only with the Premium Tier subscription of Stackdriver.For the installation instructions, see Installing the Monitoring Agent
For information about the service tiers for Stackdriver accounts, see Service tiers for Stackdriver accounts
- Configure and run the ETL. For more information, see Google Cloud Platform - GCP API ETL.
- Verify that the metrics are displayed in the Workspace.
Performance metrics from the agent
The following table contains the performance metrics that you get after installing the Monitoring Agent and enabling the Stackdriver monitoring. The following metric type names must be prefixed with agent.googleapis.com/. That prefix has been omitted so that the names fit within the table.
metric | Display name (Google Cloud metric type) | Operating system | Statistic | DataType | Sub metric label key | Sub metric label value | Sub resource label key | Reducer | GroupBy Fields |
|---|---|---|---|---|---|---|---|---|---|
BYFS_USED_SPACE_PCT | Disk utilization (disk/percent_used) | Both | ALIGN_MEAN | DOUBLE | State | State | device Devices that end with fs are excluded. | ||
BYFS_USED | Disk usage (disk/bytes_used) | Linux | ALIGN_MEAN | DOUBLE | State | Used | device Devices that end with fs are excluded. | ||
BYFS_FREE | Disk usage (disk/bytes_used) | Linux | ALIGN_MEAN | DOUBLE | State | Free | device Devices that end with fs are excluded. | ||
BYFS_SIZE | Disk usage (disk/bytes_used) | Linux | ALIGN_MEAN | DOUBLE | State | (Used + Free) | device Devices that end with fs are excluded. | ||
TOTAL_FS_USED | Disk usage (disk/bytes_used) | Linux | ALIGN_MEAN | DOUBLE | State | Used | device Devices that end with fs are excluded. | ||
TOTAL_FS_FREE | Disk usage (disk/bytes_used) | Linux | ALIGN_MEAN | DOUBLE | State | Free | device Devices that end with fs are excluded. | ||
TOTAL_FS_SIZE | Disk usage (disk/bytes_used) | Linux | ALIGN_MEAN | DOUBLE | State | (Used + Free) | device Devices that end with fs are excluded. | ||
TOTAL_FS_UTIL | Disk usage (disk/bytes_used) | Linux | ALIGN_MEAN | DOUBLE | State | (Used / (Used + Free)) | device Devices that end with fs are excluded. | ||
BYDISK_SERVICE_TIME | Disk average I/O time (disk/io_time) | Linux | ALIGN_DELTA | INT64 | device Devices that do not end with a digit are excluded. | ||||
BYDISK_PHYS_IO_READ_RATE | Disk operations (disk/operation_count) | Linux | ALIGN_RATE | DOUBLE | Direction | Read | device Devices that do not end with a digit are excluded. | ||
BYDISK_PHYS_IO_WRITE_RATE | Disk operations (disk/operation_count) | Linux | ALIGN_RATE | DOUBLE | Direction | Write | device Devices that do not end with a digit are excluded. | ||
BYDISK_PHYS_IO_RATE | Disk operations (disk/operation_count) | Linux | ALIGN_RATE | DOUBLE | (Read + Write) | device Devices that do not end with a digit are excluded. | |||
DISK_IO_BLOCKED | Disk pending operations (disk/pending_operations) | Linux | ALIGN_RATE | DOUBLE | device | ||||
MEM_REAL_USED | Memory utilization (memory/percent_used) TOTAL_REAL_MEM | Both | ALIGN_MEAN | DOUBLE | State | (Used + Slab) | |||
MEM_CACHED | Memory utilization (memory/percent_used) TOTAL_REAL_MEM | Both | ALIGN_MEAN | DOUBLE | State | Cached | |||
MEM_BUFFERED | Memory utilization (memory/percent_used) TOTAL_REAL_MEM | Both | ALIGN_MEAN | DOUBLE | State | Buffered | |||
MEM_FREE | Memory utilization (memory/percent_used) TOTAL_REAL_MEM | Both | ALIGN_MEAN | DOUBLE | State | (Used + Slab + Cached + Buffered) | |||
MEM_REAL_UTIL | Memory utilization (memory/percent_used) | Both | ALIGN_MEAN | DOUBLE | State | (Used + Slab) | |||
MEM_USED | Memory utilization (memory/percent_used) TOTAL_REAL_MEM | Both | ALIGN_MEAN | DOUBLE | State | (Used + Slab + Cached + Buffered) | |||
MEM_UTIL | Memory utilization (memory/percent_used) | Both | ALIGN_MEAN | DOUBLE | State | (Used + Slab + Cached + Buffered) | |||
MEM_UTIL_CACHED | Memory utilization (memory/percent_used) | Both | ALIGN_MEAN | DOUBLE | State | Cached | |||
SWAP_SPACE_USED | Swap usage (swap/bytes_used) | Both | ALIGN_MEAN | DOUBLE | State | Used | REDUCE_SUM | resource.labels.instance_id | |
SWAP_SPACE_FREE | Swap usage (swap/bytes_used) | Both | ALIGN_MEAN | DOUBLE | State | Free | REDUCE_SUM | resource.labels.instance_id | |
SWAP_SPACE_TOT | Swap usage (swap/bytes_used) | Both | ALIGN_MEAN | DOUBLE | REDUCE_SUM | resource.labels.instance_id | |||
SWAP_SPACE_UTIL | Swap usage (swap/bytes_used) Swap usage (swap/bytes_used) | Both | ALIGN_MEAN ALIGN_MEAN | DOUBLE DOUBLE | State | Used | REDUCE_SUM REDUCE_SUM | resource.labels.instance_id resource.labels.instance_id | |
NET_CONNECTIONS_CURRENT | TCP connections (network/tcp_connections) | Both | ALIGN_MEAN | DOUBLE | TCP_state | Established | |||
CPU_UTIL_USER | CPU utilization (cpu/utilization) | Linux | ALIGN_MEAN | DOUBLE | CPU_state | User | |||
CPU_UTIL_SYSTEM | CPU utilization (cpu/utilization) | Linux | ALIGN_MEAN | DOUBLE | CPU_state | System | |||
CPU_UTIL_IDLE | CPU utilization (cpu/utilization) | Both | ALIGN_MEAN | DOUBLE | CPU_state | Idle | |||
CPU_UTIL_WAIT | CPU utilization (cpu/utilization) | Linux | ALIGN_MEAN | DOUBLE | CPU_state | Wait | |||
CPU_UTIL_GUEST | CPU utilization (cpu/utilization) | Windows | ALIGN_MEAN | DOUBLE | CPU_state | Used |
For more information, see theGoogle Cloud Agent metricsdocumentation.