Moviri Integrator for BMC Helix Capacity Optimization - HP Reporter
All data collected can be analyzed using BMC Helix Continuous Optimization tool, in order to:
- Analyze resources usage
- Perform trend analysis and forecast analysis
- Gain at-a-glance control over IT infrastructure capacity with BMC Helix Continuous Optimization Virtual Farm Views
- Perform capacity optimization and planning
The connector supports configuration parameters that allow systems and metric collection filtering, historical recovery, limiting data volume processed in a single run, specifying data resolution and many other settings.
The documentation is targeted at BMC Helix Continuous Optimization administrators, in charge of configuring and monitoring the integration between BMC Helix Continuous Optimization and HP Reporter.
Requirements
Supported versions of data source software
HP Reporter 3.80 Eng
HP Reporter 3.90 Eng
HP Reporter 4.00 Eng
In order to collect data and relationships between IBM LPARs and Frames, the following product needs to be installed on monitored LPARs: HP Operations Smart Plug-in for Virtualization Infrastructure for HP Operations Manager
Supported configurations of data source software
HP Reporter can support various DBMSs for storing data; "Moviri - HP Reporter Extractor" supports the following:
- Oracle Database 11g
Microsoft SQL Server 2008 R2, 2012
Datasource Check and Configuration
Preparing to connect to the data source software
As a first step, please ensure that the correct version of HP Reporter is installed. You can perform this check from the "About Reporter" section of the Console:
Please also ensure that the DBMS version is compliant with the aforementioned list.
By default, HP Reporter is configured to permanently store in its database the data that is collected from agents only for a set of predefined metric lists:
- APPLICATION
- GLOBAL
- SYSDOWNTIME
- TRANSACTIONS
- UPTIME
These metric lists include only a subset of the metrics that can be imported by "Moviri Integrator for BMC Helix Continuous Optimization - HP Reporter". Please refer to HP Reporter official documentation for additional information.
In order to properly set up the HP Reporter data source for integration in CO, the following set of metric lists should be created (names are suggested):
- CPIT_APPLICATION (metric class "Application")
- CPIT_DISK (metric class "Disk")
- CPIT_FILESYSTEM (metric class "Filesystem")
- CPIT_GLOBAL (metric class "Global")
- CPIT_LOGICAL (metric class "Logical")
- CPIT_NETWORK (metric class "Netif")
And only for AIX Logic systems, two metric classes enabled by deploying on the monitored host HP Operations Smart Plug-in for Virtualization Infrastructure for HP Operations Manager
- CPIT_FRAME_CONF (metric class "Frame conf")
- CPIT_LPAR_CONF (metric class "Lpar conf")
Access to the HP Reporter console is needed to perform the above operations.
In HP Reporter, you can either create a new metric list as a copy of an existing one, or from scratch. Please refer to Section 9 for the full range of CO-mapped metrics.
After the creation of the metric lists, they need to be added to Discovered Systems > Metric Lists (by drag and drop); eventually Run All button must be pushed in order to build proper tables inside Reporter database.
The BMC Helix Continuous Optimization ETL should access the data source by means of a database user having read-only access rights on the schema (and therefore, the tables) containing the data to be collected. The database user should be created directly on the DBMS (by the Database Administrator), and not from the HP Reporter Console.
The procedure, according to HP Reporter manual, should proceed as lsited below for each new metric list:
- Add a new Metric List
- Define properties for the newly created metric list, as depicted in the following figure:
- Name.
- Source Class.
- Data resolution.
- Data retention.
- Data sources.
- Activate data gathering for the defined metrics list on all available systems
This is a summary of the configuration described in this section. The data resolution setting can be customized according to specific reporting needs, in which case one ETL task for each data resolution is to be set up.
Name | CPIT_APPLICATION | CPIT_GLOBAL | CPIT_DISK | CPIT_FILESYSTEM | CPIT_FRAME_CONF | CPIT_LPAR_CONF | CPIT_LOGICAL | CPIT_NETWORK |
Source class | APPLICATION | GLOBAL | DISK | FILESYSTEM | FRAME_CONF | LPAR_CONF | LOGICAL | NETIF |
Data resolution | 1 hr | 1 hr | 1 hr | 1 hr | 1 hr | 1 hr | 1 hr | 1 hr |
Data sources | * | * | * | * | * | * | * | * |
For the full range of metrics that can be added to the above metric lists please refer to section "Configuration and Performance/Business Metrics Mapping", subsection "Configuration Metrics".
Connector configuration
Connector configuration attributes
There are three sections (tabs) in the ETL run configuration page where the behavior of "Moviri - HP Reporter Extractor" can be specified:
- HP Reporter - Data selection
- HP Reporter - Settings
- Connection parameters
HP Reporter - Data selection
Property Name | Type | Required? | Default value | Description |
---|---|---|---|---|
Collect Global table data | Boolean | Yes | true | If true extract data from GLOBAL table, else skip table |
Global table name | String | Yes | GLOBAL | Metric list containing Global metrics (suggested additional metric list is CPIT_GLOBAL, see Section 9) |
Collect Disk table data | Boolean | Yes | true | If true extract data from DISK table, else skip table |
Disk table name | String | Yes | DISK | Metric list containing Disk metrics (suggested additional metric lists are CPIT_DISK, see Section 9 |
Collect Filesystem table data | Boolean | Yes | true | If true extract data from FILESYSTEM table, else skip table |
Filesystem table name | String | Yes | FILESYSTEM | Metric list containing Filesystems metrics (suggested additional metric lists are CPIT_FILESYSTEM, Section 9 |
Collect Network table data | Boolean | Yes | true | If true extract data from Network Interfaces table, else skip table |
Network table name | String | Yes | NETWORK | Metric list containing Network Interfaces metrics (suggested additional metric list is CPIT_NETWORK, see Section 9) |
Collect Application table data | Boolean | Yes | true | If true extract data from APPLICATION table, else skip table |
Application table name | String | Yes | APPLICATION | Metric list containing Application-related metrics (suggested additional metric list is CPIT_APPLICATION, see Section 9) |
Collect Logical systems table data | Boolean | Yes | true | If true extract data from Logical systems table, else skip table |
Logical systems table name | String | Yes | LOGICAL | Metric list containing Logical systems metrics (suggested additional metric list is CPIT_LOGICAL, see Section 9) |
Collect LPAR table data | Boolean | Yes | true | If true extract data from Network Interfaces table, else skip table |
LPAR table name | String | Yes | LPAR | Metric list containing LPAR relationships (suggested additional metric list is CPIT_LPAR, see Section 9) |
Collect Frame configuration table data | Boolean | Yes | true | If true extract data from Frame table, else skip table |
Frame configuration table | String | Yes | FRAME_CONF | Metric list containing Frame metrics (suggested additional metric lists are CPIT_ FRAME_CONF, see Section 9 |
Devices White List(separator ';' SQL wildcards allowed) | String | No |
| Semicolon-separated list of servers whose data is going to be imported, SQL wildcard "%" and "_" can be used . Empty list means all servers. |
Devices Black List(separator ';' SQL wildcards allowed) | String | No |
| Semicolon-separated list of servers whose data is not going to be imported, SQL wildcard "%" and "_" can be used. Empty list means none. |
HP Reporter - Settings
Property Name | Type | Required? | Default value | Description |
---|---|---|---|---|
Database type (ORACLE or MSSQLSERVER) | String | Yes |
| Reporter export database type (Oracle or MS SQL Server) |
Database table prefix | String | No |
| If more than one Reporter schema exist, specify the correct DB schema to be used. |
Default last counter (YYYY-MM-DD HH24:MI:SS) | String | Yes |
| Value to be used when lastcounter is not defined for a table in HP Reporter database (Suggested: one day ago). |
Max days to extract | String | No |
| In order to limit data volume, this property sets the maximum number of days to be extracted (starting from lastcounter). |
Extract resolution (in minutes/ORIGINAL) | String | Yes | 60 | Data will be aggregated into samples with this granularity (expressed in minutes; otherwise use the string ORIGINAL to use data source native resolution) |
Historical data extraction
In order to perform a historical recovery of data, please use the default, "lastcounter"-based mechanism of CO, i.e. change the current value of the desired lastcounter.
If the ETL task is newly created and has no lastcounter defined, the "default lastcounter" can be used
Supported Platforms
For all metrics imported by the ETL and described in this section, the following apply:
- Supported platforms: Standalone OS Instance, AIX Power VM Hosts and LPARs, Solaris Containers
- Supported BMC Helix Continuous Optimization Entity Type: System (Generic), Solaris Zone, Solaris Virtual Host, AIX LPAR, AIX SPLPAR, AIX Virtual Host
Troubleshooting
Missing data can represent a frequent problem; in this case a WARNING status would be associated to the ETL task. This can be due to a number of scenarios, among which:
- The data is not present on the HP Reporter data source. In order to verify this, it is suggested to run the HP Reporter reports against the considered time range.
Figure 23: Generation of HP Reporter Reports
- The data is available on HP Reporter, but the ETL is not able to extract it. This could be the case when the ETL task is executed before new data (since the last collection) is available on the data source. The last counter, which contains the highest time stamp collected in the previous run for that table (and is used as a time threshold for ignoring old data), is higher than the data to extract timestamp.
The lack of extracted data (see the following log example) indicates that new data might not be available in HP Reporter. In this case, the previous condition applies and the recommendation is to run the ETL at a later date.
Moreover, it is strongly recommended to avoid creating an Integration Studio ETL module with the same name as this ETL (HPReporterME), which would likely cause unexpected behavior.
Configuration and Performance Metrics Mapping
Configuration Metrics
The following configuration metrics are imported into CO:
Metrics with * were previously custom, remapped as standard after v 3.4.00
HP Reporter Metric Name | BMC Helix Continuous Optimization Metric | HP Reporter metric class | Factor | Entity |
OSNAME | OS_TYPE | SYSTEMS | Generic | |
OSRELEASE | OS_VER | SYSTEMS | Generic | |
MACHINETYPE | HW_MODEL | SYSTEMS | Generic | |
MEMORY | TOTAL_REAL_MEM | SYSTEMS | 1048576 | Generic |
SWAP | SWAP_SPACE_TOT | SYSTEMS | 1024 | Generic |
IPADDRESS | NET_IP_ADDRESS | SYSTEMS | Generic | |
DISKS | DISK_NUM | SYSTEMS |
| Generic |
NETWORKS | NETIF_NUM | SYSTEMS |
| Generic |
CPUS | CPU_NUM | SYSTEMS |
| Generic |
| ||||
GBL_CPU_CLOCK | CPU_MHZ | GLOBAL | Generic | |
GBL_SWAP_SPACE_AVAIL | SWAP_SPACE_TOT | GLOBAL | 1048576 | Generic |
| ||||
HMC_NAME | HMC_NAME* | FRAME_CONF |
| AIX VH |
FRAME_IP | NET_IP_ADDRESS | FRAME_CONF |
| AIX VH |
FRAME_MODEL_TYPE | HW_MODEL | FRAME_CONF |
| AIX VH |
| ||||
BYLS_CPU_ENTL | CPU_ENTC | LOGICAL |
| LPAR, Zone |
BYLS_CPU_SHARES_PRIO | CPU_SHARES* | LOGICAL |
| LPAR, Zone |
BYLS_CPU_MT_ENABLED | LP_SMT | LOGICAL |
| LPAR |
BYLS_CPU_ENTL_MAX | LP_CAPACITY_MAX | LOGICAL |
| LPAR |
BYLS_CPU_ENTL_MIN | LP_CAPACITY_MIN | LOGICAL |
| LPAR |
BYLS_LS_SHARED | LS_SHARED_C | LOGICAL |
| LPAR |
BYLS_LS_MODE | LP_MODE | LOGICAL |
| LPAR, Zone |
BYLS_LS_STATE | LP_STATUS | LOGICAL |
| LPAR |
BYLS_LS_TYPE | LP_TYPE | LOGICAL |
| LPAR |
BYLS_CPU_ENTL_MIN | ZN_CPU_ENTC_MIN* | LOGICAL |
| Zone |
BYLS_LS_STATE | ZN_STATUS* | LOGICAL |
| Zone |
BYLS_IP_ADDRESS | NET_IP_ADDRESS | LOGICAL |
| LPAR, Zone |
BYLS_LS_HOSTNAME | HOST_NAME | LOGICAL |
| LPAR, Zone |
BYLS_MEM_SWAP | SWAP_SPACE_TOT | LOGICAL | 1048576 | Zone |
BYLS_MEM_ENTL | TOTAL_REAL_MEM | LOGICAL | 1048576 | LPAR, Zone |
BYLS_NUM_CPU | CPU_NUM | LOGICAL |
| LPAR, Zone |
BYLS_NUM_DISK | DISK_NUM | LOGICAL |
| LPAR, Zone |
BYLS_NUM_NETIF | NETIF_NUM | LOGICAL |
| LPAR, Zone |
| ||||
BYNETIF_NET_SPEED | BYIF_BANDWIDTH | NETIF | 1000000 | Generic |
Performance Metrics
The ETL connects to the Reporter Warehouse instance and extracts data for metrics belonging to specific HP Reporter metrics’ classes. These classes of performance metrics are populated by HP agents and refer to:
- Generic OS instances
- Solaris Zones
- Solaris Virtual Host
- AIX LPAR
- AIX Virtual Host
The full list of known classes and metrics, and the mapping between HP Reporter metrics and BMC Helix Continuous Optimization metrics is reported in the listing below.
Reporter Class | Reporter Metric Name | BMC Helix Continuous Optimization dataset | BMC Helix Continuous Optimization Metric | Factor | Entity |
GLOBAL | GBL_CPU_TOTAL_UTIL | SYSGLB | CPU_UTIL | 0.01 | Generic |
GLOBAL | GBL_CPU_USER_MODE_UTIL | SYSGLB | CPU_UTIL_USER | 0.01 | Generic |
GLOBAL | GBL_CPU_SYS_MODE_UTIL | SYSGLB | CPU_UTIL_SYSTEM | 0.01 | Generic |
GLOBAL | GBL_DISK_UTIL | SYSGLB | DISK_UTIL | 0.01 | Generic |
GLOBAL | GBL_RUN_QUEUE | SYSGLB | CPU_RUN_QUEUE | Generic | |
GLOBAL | GBL_CPU_WAIT_UTIL | SYSGLB | CPU_UTIL_WAIO | 0.01 | Generic |
GLOBAL | GBL_DISK_UTIL_PEAK | SYSGLB | DISK_UTIL_PEAK | 0.01 | Generic |
GLOBAL | GBL_SWAP_SPACE_USED | SYSGLB | SWAP_SPACE_USED | Generic | |
GLOBAL | GBL_SWAP_SPACE_UTIL | SYSGLB | SWAP_SPACE_UTIL | 0.01 | Generic |
GLOBAL | GBL_DISK_PHYS_IO_RATE | SYSGLB | DISK_PHYS_IO_RATE | Generic | |
GLOBAL | GBL_DISK_PHYS_BYTE_RATE | SYSGLB | DISK_TRANSFER_RATE | Generic | |
GLOBAL | GBL_NET_IN_PACKET_RATE | SYSGLB | NET_IN_PKT_RATE | Generic | |
GLOBAL | GBL_NET_OUT_PACKET_RATE | SYSGLB | NET_OUT_PKT_RATE | Generic | |
GLOBAL | GBL_MEM_UTIL | SYSGLB | MEM_UTIL | 0.01 | Generic |
GLOBAL | GBL_MEM_USER_UTIL | SYSGLB | MEM_UTIL_USER | 0.01 | Generic |
GLOBAL | GBL_MEM_SYS_UTIL | SYSGLB | MEM_UTIL_SYSTEM | 0.01 | Generic |
GLOBAL | GBL_MEM_FREE | SYSGLB | MEM_FREE | Generic | |
GLOBAL | GBL_MEM_PAGE_REQUEST_RATE | SYSGLB | MEM_PAGE_RATE | Generic | |
GLOBAL | GBL_MEM_PAGEIN_RATE | SYSGLB | MEM_PAGE_IN_RATE | Generic | |
GLOBAL | GBL_MEM_PAGEOUT_RATE | SYSGLB | MEM_PAGE_OUT_RATE | Generic | |
GLOBAL | GBL_MEM_ACTIVE_VIRT_UTIL | SYSGLB | MEM_ACTIVE_VIRTUAL | Generic | |
GLOBAL | GBL_FS_SPACE_UTIL_PEAK | SYSGLB | LDISK_USED_SPACE_PCT_PEAK |
| Generic |
| |||||
FILESYSTEM | FS_DIRNAME | SYSDSK | <ldisk_name> | Generic | |
FILESYSTEM | FS_SPACE_UTIL | SYSDSK | BYLDISK_USED_SPACE_PCT | 0.01 | Generic |
| |||||
DISK | BYDSK_UTIL | SYSDSK | BYDISK_UTIL | 0.01 | Generic |
DISK | BYDSK_DEVNAME | SYSDSK | <objname> | Generic | |
DISK | BYDSK_PHYS_BYTE_RATE | SYSDSK | BYDISK_TRANSFER_BYTES | Generic | |
DISK | BYDSK_PHYS_IO_RATE | SYSDSK | BYDISK_TRANSFER_RATE | Generic | |
| |||||
NETWORK | BYNETIF_OUT_BYTE_RATE | SYSNET | BYIF_OUT_BYTE_RATE | Generic | |
NETWORK | BYNETIF_IN_BYTE_RATE | SYSNET | BYIF_IN_BYTE_RATE | Generic | |
NETWORK | BYNETIF_NAME | SYSNET | <objname> | Generic | |
NETWORK | BYNETIF_IN_PACKET_RATE | SYSNET | BYIF_IN_PKT_RATE | Generic | |
NETWORK | BYNETIF_OUT_PACKET_RATE | SYSNET | BYIF_OUT_PKT_RATE | Generic | |
NETWORK | BYNETIF_ERROR_RATE | SYSNET | BYIF_ERROR_RATE | Generic | |
| |||||
APPLICATION | APP_NAME | SYSPROC | <objname> | Generic | |
APPLICATION | APP_CPU_TOTAL_TIME | SYSPROC | BYAPP_CPU_TIME | Generic | |
APPLICATION | APP_CPU_TOTAL_UTIL | SYSPROC | BYAPP_CPU_UTIL | 0.01 | Generic |
APPLICATION | APP_MEM_VIRT | SYSPROC | BYAPP_MEM_VIRTUAL_SIZE | 1024 | Generic |
| |||||
LOGICAL | BYLS_CPU_ENTL_UTIL | SYSGLB | CPU_ENT_UTIL | 0.01 | Zone, LPAR |
LOGICAL | BYLS_CPU_TOTAL_UTIL | SYSGLB | CPU_UTIL | 0.01 | Zone, LPAR |
LOGICAL | BYLS_CPU_PHYS_IDLE_MODE_UTIL | SYSGLB | CPU_UTIL_IDLE | 0.01 | LPAR |
LOGICAL | BYLS_CPU_PHYS_SYS_MODE_UTIL | SYSGLB | CPU_UTIL_SYSTEM | 0.01 | LPAR |
LOGICAL | BYLS_CPU_PHYS_USER_MODE_UTIL | SYSGLB | CPU_UTIL_USER | 0.01 | LPAR |
LOGICAL | BYLS_CPU_PHYS_WAIT_MODE_UTIL | SYSGLB | CPU_UTIL_WAIO | 0.01 | LPAR |
LOGICAL | BYLS_CPU_PHYSC | SYSGLB | CPU_ENTC_CONS |
| Zone, LPAR |
LOGICAL | BYLS_RUN_QUEUE | SYSGLB | CPU_RUN_QUEUE |
| LPAR |
LOGICAL | BYLS_VCSWITCH_RATE | SYSGLB | CTX_SWITCH_RATE |
| Zone, LPAR |
LOGICAL | BYLS_MEM_CONS | SYSGLB | MEM_CONSUMED | 1048576 | Zone, LPAR |
LOGICAL | BYLS_MEM_ENTL_UTIL | SYSGLB | MEM_UTIL | 0.01 | Zone, LPAR |
LOGICAL | BYLS_MEM_SWAP_USED | SYSGLB | SWAP_SPACE_USED | 1048576 | Zone |
LOGICAL | BYLS_MEM_SWAP_UTIL | SYSGLB | SWAP_SPACE_UTIL | 0.01 | Zone |
| |||||
FRAME_CONF | FRAME_PROC_CONFIG | SYSGLB | CPU_NUM |
| AIX VH |
FRAME_CONF | FRAME_PROC_AVAIL | SYSGLB | CPU_MAX_PROCESSORS |
| AIX VH |
FRAME_CONF | FRAME_MEM_CONFIG | SYSGLB | TOTAL_EST_REAL_MEM | 1048576 | AIX VH |
FRAME_CONF | FRAME_MEM_AVAIL | SYSGLB | TOTAL_REAL_MEM | 1048576 | AIX VH |
Inside all metric lists created in HP Reporter, it is important to include the INTERVAL and GLB_INTERVAL columns, wherever applicable.
Lookup Fields
Moviri Integrator for BMC Helix Continuous Optimization connector uses standard BMC Helix Continuous Optimization ETL lookup mechanism. One single entry is used as entity identifier: Lookup field = “DEFAULT”.
Here below a lookup tables as example:
Object Relationships
The ETL assigns all imported entities under a domain (which can be configured by the user).
- Imported Relationship Types:
- “Domain to system” APP – SYS [identified by its data source name, DS_SYSNM], where APP is the domain of choice (as selected in the “Object Relationships” tab in the Run Configuration wizard)
- “System to System” SYS [VH] – SYS [GM],
- Solaris zone are connected to their container host (see Figure 26)
- AIX LPAR are connected to their frame host
A hierarchy rule associated to the ETL is automatically created by BMC Helix Continuous Optimization to process the Object-Relationship data.
Comments
Log in or register to comment.