Page tree
Skip to end of metadata
Go to start of metadata

"Moviri Integrator for TrueSight Capacity Optimization – HP Reporter" allows collecting information from HP Reporter tool and it is able to extract and summarize resources performance data, configuration data and relationships between infrastructure entities.


All data collected can be analyzed using TrueSight Capacity Optimization tool, in order to:

  • Analyze resources usage
  • Perform trend analysis and forecast analysis
  • Gain at-a-glance control over IT infrastructure capacity with TrueSight Capacity Optimization Virtual Farm Views
  • Perform capacity optimization and planning


The connector supports configuration parameters that allow systems and metric collection filtering, historical recovery, limiting data volume processed in a single run, specifying data resolution and many other settings.
The documentation is targeted at TrueSight Capacity Optimization administrators, in charge of configuring and monitoring the integration between TrueSight Capacity Optimization and HP Reporter.

Requirements

Supported versions of data source software

  • HP Reporter 3.80 Eng

  • HP Reporter 3.90 Eng

  • HP Reporter 4.00 Eng

In order to collect data and relationships between IBM LPARs and Frames, the following product needs to be installed on monitored LPARs: HP Operations Smart Plug-in for Virtualization Infrastructure for HP Operations Manager

Supported configurations of data source software

HP Reporter can support various DBMSs for storing data; "Moviri - HP Reporter Extractor" supports the following:

  • Oracle Database 11g
  • Microsoft SQL Server 2008 R2, 2012

Installation

Downloading the additional package

ETL Module is made available in the form of an additional component, which you may download from BMC electronic distribution site (EPD) or retrieve from your content media.

Installing the additional package

To install the connector in the form of a TrueSight Capacity Optimization additional package, refer to Performing system maintenance tasks instructions.

Datasource Check and Configuration

Preparing to connect to the data source software

As a first step, please ensure that the correct version of HP Reporter is installed. You can perform this check from the "About Reporter" section of the Console:

Please also ensure that the DBMS version is compliant with the aforementioned list.
By default, HP Reporter is configured to permanently store in its database the data that is collected from agents only for a set of predefined metric lists:

  • APPLICATION
  • GLOBAL
  • SYSDOWNTIME
  • TRANSACTIONS
  • UPTIME

These metric lists include only a subset of the metrics that can be imported by "Moviri Integrator for TrueSight Capacity Optimization - HP Reporter". Please refer to HP Reporter official documentation for additional information.
In order to properly set up the HP Reporter data source for integration in CO, the following set of metric lists should be created (names are suggested):

  • CPIT_APPLICATION (metric class "Application")
  • CPIT_DISK (metric class "Disk")
  • CPIT_FILESYSTEM (metric class "Filesystem")
  • CPIT_GLOBAL (metric class "Global")
  • CPIT_LOGICAL (metric class "Logical")
  • CPIT_NETWORK (metric class "Netif")

And only for AIX Logic systems, two metric classes enabled by deploying on the monitored host HP Operations Smart Plug-in for Virtualization Infrastructure for HP Operations Manager

  • CPIT_FRAME_CONF (metric class "Frame conf")
  • CPIT_LPAR_CONF (metric class "Lpar conf")


Access to the HP Reporter console is needed to perform the above operations.
In HP Reporter, you can either create a new metric list as a copy of an existing one, or from scratch. Please refer to Section 9 for the full range of CO-mapped metrics.
After the creation of the metric lists, they need to be added to Discovered Systems > Metric Lists (by drag and drop); eventually Run All button must be pushed in order to build proper tables inside Reporter database.
The TrueSight Capacity Optimization ETL should access the data source by means of a database user having read-only access rights on the schema (and therefore, the tables) containing the data to be collected. The database user should be created directly on the DBMS (by the Database Administrator), and not from the HP Reporter Console.
The procedure, according to HP Reporter manual, should proceed as lsited below for each new metric list:

  1. Add a new Metric List


  2. Define properties for the newly created metric list, as depicted in the following figure:
    1. Name.
    2. Source Class.
    3. Data resolution.
    4. Data retention.
    5. Data sources.
    Then, add metrics from Data sources according to the list of suggested metrics (see table at the end of current section).


  3. Activate data gathering for the defined metrics list on all available systems


 


This is a summary of the configuration described in this section. The data resolution setting can be customized according to specific reporting needs, in which case one ETL task for each data resolution is to be set up.

Name

CPIT_APPLICATION

CPIT_GLOBAL

CPIT_DISK

CPIT_FILESYSTEM

CPIT_FRAME_CONF

CPIT_LPAR_CONF

CPIT_LOGICAL

CPIT_NETWORK

Source class

APPLICATION

GLOBAL

DISK

FILESYSTEM

FRAME_CONF

LPAR_CONF

LOGICAL

NETIF

Data resolution

1 hr

1 hr

1 hr

1 hr

1 hr

1 hr

1 hr

1 hr

Data sources

*

*

*

*

*

*

*

*

 

For the full range of metrics that can be added to the above metric lists please refer to section "Configuration and Performance/Business Metrics Mapping", subsection "Configuration Metrics".

Connector configuration

Connector configuration attributes

There are three sections (tabs) in the ETL run configuration page where the behavior of "Moviri - HP Reporter Extractor" can be specified:

  • HP Reporter - Data selection
  • HP Reporter - Settings
  • Connection parameters

HP Reporter - Data selection


 

Property Name

Type

Required?

Default value

Description

Collect Global table data

Boolean

Yes

true

If true extract data from GLOBAL table, else skip table

Global table name

String

Yes

GLOBAL

Metric list containing Global metrics (suggested additional metric list is CPIT_GLOBAL, see Section 9)

Collect Disk table data

Boolean

Yes

true

If true extract data from DISK table, else skip table

Disk table name

String

Yes

DISK

Metric list containing Disk metrics (suggested additional metric lists are CPIT_DISK, see Section 9

Collect Filesystem table data

Boolean

Yes

true

If true extract data from FILESYSTEM table, else skip table

Filesystem table name

String

Yes

FILESYSTEM

Metric list containing Filesystems metrics (suggested additional metric lists are CPIT_FILESYSTEM, Section 9

Collect Network table data

Boolean

Yes

true

If true extract data from Network Interfaces table, else skip table

Network table name

String

Yes

NETWORK

Metric list containing Network Interfaces metrics (suggested additional metric list is CPIT_NETWORK, see Section 9)

Collect Application table data

Boolean

Yes

true

If true extract data from APPLICATION table, else skip table

Application table name

String

Yes

APPLICATION

Metric list containing Application-related metrics (suggested additional metric list is CPIT_APPLICATION, see Section 9)

Collect Logical systems table data

Boolean

Yes

true

If true extract data from Logical systems table, else skip table

Logical systems table name

String

Yes

LOGICAL

Metric list containing Logical systems metrics (suggested additional metric list is CPIT_LOGICAL, see Section 9)

Collect LPAR table data

Boolean

Yes

true

If true extract data from Network Interfaces table, else skip table

LPAR table name

String

Yes

LPAR

Metric list containing LPAR relationships (suggested additional metric list is CPIT_LPAR, see Section 9)

Collect Frame configuration table data

Boolean

Yes

true

If true extract data from Frame table, else skip table

Frame configuration table

String

Yes

FRAME_CONF

Metric list containing Frame metrics (suggested additional metric lists are CPIT_ FRAME_CONF, see Section 9

Devices White List(separator ';' SQL wildcards allowed)

String

No

 

Semicolon-separated list of servers whose data is going to be imported, SQL wildcard "%" and "_" can be used . Empty list means all servers.

Devices Black List(separator ';' SQL wildcards allowed)

String

No

 

Semicolon-separated list of servers whose data is not going to be imported, SQL wildcard "%" and "_" can be used. Empty list means none.

 

HP Reporter - Settings


 

Property Name

Type

Required?

Default value

Description

Database type (ORACLE or MSSQLSERVER)

String

Yes

 

Reporter export database type (Oracle or MS SQL Server)

Database table prefix

String

No

 

If more than one Reporter schema exist, specify the correct DB schema to be used.

Default last counter (YYYY-MM-DD HH24:MI:SS)

String

Yes

 

Value to be used when lastcounter is not defined for a table in HP Reporter database (Suggested: one day ago).

Max days to extract

String

No

 

In order to limit data volume, this property sets the maximum number of days to be extracted (starting from lastcounter).

Extract resolution (in minutes/ORIGINAL)

String

Yes

60

Data will be aggregated into samples with this granularity (expressed in minutes; otherwise use the string ORIGINAL to use data source native resolution)

Historical data extraction

In order to perform a historical recovery of data, please use the default, "lastcounter"-based mechanism of CO, i.e. change the current value of the desired lastcounter.
If the ETL task is newly created and has no lastcounter defined, the "default lastcounter" can be used

Supported Platforms

For all metrics imported by the ETL and described in this section, the following apply:

  • Supported platforms: Standalone OS Instance, AIX Power VM Hosts and LPARs, Solaris Containers
  • Supported TrueSight Capacity Optimization Entity Type: System (Generic), Solaris Zone, Solaris Virtual Host, AIX LPAR, AIX SPLPAR, AIX Virtual Host

 

Troubleshooting

 Missing data can represent a frequent problem; in this case a WARNING status would be associated to the ETL task. This can be due to a number of scenarios, among which:

  • The data is not present on the HP Reporter data source. In order to verify this, it is suggested to run the HP Reporter reports against the considered time range.


Figure 23: Generation of HP Reporter Reports

  • The data is available on HP Reporter, but the ETL is not able to extract it. This could be the case when the ETL task is executed before new data (since the last collection) is available on the data source. The last counter, which contains the highest time stamp collected in the previous run for that table (and is used as a time threshold for ignoring old data), is higher than the data to extract timestamp.

The lack of extracted data (see the following log example) indicates that new data might not be available in HP Reporter. In this case, the previous condition applies and the recommendation is to run the ETL at a later date.



 

 
Moreover, it is strongly recommended to avoid creating an Integration Studio ETL module with the same name as this ETL (HPReporterME), which would likely cause unexpected behavior.

Configuration and Performance Metrics Mapping

Configuration Metrics

The following configuration metrics are imported into CO:

Metrics with * were previously custom, remapped as standard after v 3.4.00

 

HP Reporter Metric Name

TrueSight Capacity Optimization Metric

HP Reporter metric class

Factor

Entity

OSNAME

OS_TYPE

SYSTEMS

 

Generic

OSRELEASE

OS_VER

SYSTEMS

 

Generic

MACHINETYPE

HW_MODEL 

SYSTEMS

 

Generic

MEMORY

TOTAL_REAL_MEM

SYSTEMS

1048576

Generic

SWAP

SWAP_SPACE_TOT

SYSTEMS

1024

Generic

IPADDRESS

NET_IP_ADDRESS 

SYSTEMS

 

Generic

DISKS

DISK_NUM 

SYSTEMS

 

Generic

NETWORKS

NETIF_NUM 

SYSTEMS

 

Generic

CPUS

CPU_NUM 

SYSTEMS

 

Generic

  

GBL_CPU_CLOCK

CPU_MHZ 

GLOBAL

 

Generic

GBL_SWAP_SPACE_AVAIL

SWAP_SPACE_TOT

GLOBAL

1048576

Generic

  

HMC_NAME

HMC_NAME*

FRAME_CONF

 

AIX VH

FRAME_IP

NET_IP_ADDRESS 

FRAME_CONF

 

AIX VH

FRAME_MODEL_TYPE

HW_MODEL

FRAME_CONF

 

AIX VH

 

BYLS_CPU_ENTL

CPU_ENTC

LOGICAL

 

LPAR, Zone

BYLS_CPU_SHARES_PRIO

CPU_SHARES*
(Previously CPU_SHARES_PRIO_C)

LOGICAL

 

LPAR, Zone

BYLS_CPU_MT_ENABLED

LP_SMT

LOGICAL

 

LPAR

BYLS_CPU_ENTL_MAX

LP_CAPACITY_MAX

LOGICAL

 

LPAR

BYLS_CPU_ENTL_MIN

LP_CAPACITY_MIN

LOGICAL

 

LPAR

BYLS_LS_SHARED

LS_SHARED_C

LOGICAL

 

LPAR

BYLS_LS_MODE

LP_MODE

LOGICAL

 

LPAR, Zone

BYLS_LS_STATE

LP_STATUS

LOGICAL

 

LPAR

BYLS_LS_TYPE

LP_TYPE

LOGICAL

 

LPAR

BYLS_CPU_ENTL_MIN

ZN_CPU_ENTC_MIN*

LOGICAL

 

Zone

BYLS_LS_STATE

ZN_STATUS*

LOGICAL

 

Zone

BYLS_IP_ADDRESS

NET_IP_ADDRESS

LOGICAL

 

LPAR, Zone

BYLS_LS_HOSTNAME

HOST_NAME

LOGICAL

 

LPAR, Zone

BYLS_MEM_SWAP

SWAP_SPACE_TOT

LOGICAL

1048576

Zone

BYLS_MEM_ENTL

TOTAL_REAL_MEM

LOGICAL

1048576

LPAR, Zone

BYLS_NUM_CPU

CPU_NUM

LOGICAL

 

LPAR, Zone

BYLS_NUM_DISK

DISK_NUM

LOGICAL

 

LPAR, Zone

BYLS_NUM_NETIF

NETIF_NUM

LOGICAL

 

LPAR, Zone

  

BYNETIF_NET_SPEED

BYIF_BANDWIDTH

NETIF

1000000

Generic

Performance Metrics

The ETL connects to the Reporter Warehouse instance and extracts data for metrics belonging to specific HP Reporter metrics’ classes. These classes of performance  metrics are populated by HP agents and refer to:

  • Generic OS instances
  • Solaris Zones
  • Solaris Virtual Host
  • AIX LPAR
  • AIX Virtual Host

The full list of known classes and metrics, and the mapping between HP Reporter metrics and TrueSight Capacity Optimization metrics is reported in the listing below.

 

Reporter Class

Reporter Metric Name

TrueSight Capacity Optimization dataset

TrueSight Capacity Optimization Metric

Factor

Entity

GLOBAL

 GBL_CPU_TOTAL_UTIL

SYSGLB

CPU_UTIL

0.01

Generic

GLOBAL

 GBL_CPU_USER_MODE_UTIL

SYSGLB

CPU_UTIL_USER

0.01

Generic

GLOBAL

 GBL_CPU_SYS_MODE_UTIL

SYSGLB

CPU_UTIL_SYSTEM

0.01

Generic

GLOBAL

 GBL_DISK_UTIL

SYSGLB

DISK_UTIL

0.01

Generic

GLOBAL

 GBL_RUN_QUEUE

SYSGLB

CPU_RUN_QUEUE 

 

Generic

GLOBAL

 GBL_CPU_WAIT_UTIL

SYSGLB

CPU_UTIL_WAIO

0.01

Generic

GLOBAL

 GBL_DISK_UTIL_PEAK

SYSGLB

DISK_UTIL_PEAK

0.01

Generic

GLOBAL

 GBL_SWAP_SPACE_USED

SYSGLB

SWAP_SPACE_USED 

 

Generic

GLOBAL

 GBL_SWAP_SPACE_UTIL

SYSGLB

SWAP_SPACE_UTIL 

0.01

Generic

GLOBAL

 GBL_DISK_PHYS_IO_RATE

SYSGLB

DISK_PHYS_IO_RATE 

 

Generic

GLOBAL

 GBL_DISK_PHYS_BYTE_RATE

SYSGLB

DISK_TRANSFER_RATE 

 

Generic

GLOBAL

 GBL_NET_IN_PACKET_RATE

SYSGLB

NET_IN_PKT_RATE 

 

Generic

GLOBAL

 GBL_NET_OUT_PACKET_RATE

SYSGLB

NET_OUT_PKT_RATE 

 

Generic

GLOBAL

 GBL_MEM_UTIL

SYSGLB

MEM_UTIL

0.01

Generic

GLOBAL

 GBL_MEM_USER_UTIL

SYSGLB

MEM_UTIL_USER

0.01

Generic

GLOBAL

 GBL_MEM_SYS_UTIL

SYSGLB

MEM_UTIL_SYSTEM

0.01

Generic

GLOBAL

 GBL_MEM_FREE

SYSGLB

MEM_FREE 

 

Generic

GLOBAL

 GBL_MEM_PAGE_REQUEST_RATE

SYSGLB

MEM_PAGE_RATE 

 

Generic

GLOBAL

 GBL_MEM_PAGEIN_RATE

SYSGLB

MEM_PAGE_IN_RATE 

 

Generic

GLOBAL

 GBL_MEM_PAGEOUT_RATE

SYSGLB

MEM_PAGE_OUT_RATE 

 

Generic

GLOBAL

 GBL_MEM_ACTIVE_VIRT_UTIL

SYSGLB

MEM_ACTIVE_VIRTUAL 

 

Generic

GLOBAL

 GBL_FS_SPACE_UTIL_PEAK

SYSGLB

LDISK_USED_SPACE_PCT_PEAK

 

Generic

 

FILESYSTEM

 FS_DIRNAME

SYSDSK

<ldisk_name>

 

Generic

FILESYSTEM

 FS_SPACE_UTIL

SYSDSK

BYLDISK_USED_SPACE_PCT

0.01

Generic

 

DISK

 BYDSK_UTIL

SYSDSK

BYDISK_UTIL

0.01

Generic

DISK

 BYDSK_DEVNAME

SYSDSK

<objname>

 

Generic

DISK

 BYDSK_PHYS_BYTE_RATE

SYSDSK

BYDISK_TRANSFER_BYTES

 

Generic

DISK

 BYDSK_PHYS_IO_RATE

SYSDSK

BYDISK_TRANSFER_RATE

 

Generic

 

NETWORK

BYNETIF_OUT_BYTE_RATE

SYSNET

BYIF_OUT_BYTE_RATE

 

Generic

NETWORK

BYNETIF_IN_BYTE_RATE

SYSNET

BYIF_IN_BYTE_RATE 

 

Generic

NETWORK

BYNETIF_NAME

SYSNET

<objname>

 

Generic

NETWORK

BYNETIF_IN_PACKET_RATE

SYSNET

BYIF_IN_PKT_RATE 

 

Generic

NETWORK

BYNETIF_OUT_PACKET_RATE

SYSNET

BYIF_OUT_PKT_RATE 

 

Generic

NETWORK

BYNETIF_ERROR_RATE

SYSNET

BYIF_ERROR_RATE

 

Generic

 

APPLICATION

APP_NAME

SYSPROC

<objname>

 

Generic

APPLICATION

APP_CPU_TOTAL_TIME

SYSPROC

BYAPP_CPU_TIME

 

Generic

APPLICATION

APP_CPU_TOTAL_UTIL

SYSPROC

BYAPP_CPU_UTIL

0.01

Generic

APPLICATION

APP_MEM_VIRT

SYSPROC

BYAPP_MEM_VIRTUAL_SIZE

1024

Generic

 

LOGICAL

BYLS_CPU_ENTL_UTIL

SYSGLB

CPU_ENT_UTIL

0.01

Zone, LPAR

LOGICAL

BYLS_CPU_TOTAL_UTIL

SYSGLB

CPU_UTIL

0.01

Zone, LPAR

LOGICAL

BYLS_CPU_PHYS_IDLE_MODE_UTIL

SYSGLB

CPU_UTIL_IDLE

0.01

LPAR

LOGICAL

BYLS_CPU_PHYS_SYS_MODE_UTIL

SYSGLB

CPU_UTIL_SYSTEM

0.01

LPAR

LOGICAL

BYLS_CPU_PHYS_USER_MODE_UTIL

SYSGLB

CPU_UTIL_USER

0.01

LPAR

LOGICAL

BYLS_CPU_PHYS_WAIT_MODE_UTIL

SYSGLB

CPU_UTIL_WAIO

0.01

LPAR

LOGICAL

BYLS_CPU_PHYSC

SYSGLB

CPU_ENTC_CONS

 

Zone, LPAR

LOGICAL

BYLS_RUN_QUEUE

SYSGLB

CPU_RUN_QUEUE

 

LPAR

LOGICAL

BYLS_VCSWITCH_RATE

SYSGLB

CTX_SWITCH_RATE

 

Zone, LPAR

LOGICAL

BYLS_MEM_CONS

SYSGLB

MEM_CONSUMED

1048576

Zone, LPAR

LOGICAL

BYLS_MEM_ENTL_UTIL

SYSGLB

MEM_UTIL

0.01

Zone, LPAR

LOGICAL

BYLS_MEM_SWAP_USED

SYSGLB

SWAP_SPACE_USED

1048576

Zone

LOGICAL

BYLS_MEM_SWAP_UTIL

SYSGLB

SWAP_SPACE_UTIL

0.01

Zone

 

FRAME_CONF

FRAME_PROC_CONFIG

SYSGLB

CPU_NUM

 

AIX VH

FRAME_CONF

FRAME_PROC_AVAIL

SYSGLB

CPU_MAX_PROCESSORS

 

AIX VH

FRAME_CONF

FRAME_MEM_CONFIG

SYSGLB

TOTAL_EST_REAL_MEM

1048576

AIX VH

FRAME_CONF

FRAME_MEM_AVAIL

SYSGLB

TOTAL_REAL_MEM

1048576

AIX VH

Inside all metric lists created in HP Reporter, it is important to include the INTERVAL and GLB_INTERVAL columns, wherever applicable.

Lookup Fields

Moviri Integrator for TrueSight Capacity Optimization connector uses standard TrueSight Capacity Optimization ETL lookup mechanism. One single entry is used as entity identifier: Lookup field = “DEFAULT”.

Here below a lookup tables as example:

Object Relationships

The ETL assigns all imported entities under a domain (which can be configured by the user).

  • Imported Relationship Types:
    • “Domain to system” APP – SYS [identified by its data source name, DS_SYSNM], where APP is the domain of choice (as selected in the “Object Relationships” tab in the Run Configuration wizard)
    • “System to System” SYS [VH] – SYS [GM],
      • Solaris zone are connected to their container host (see Figure 26)
      • AIX LPAR are connected to their frame host

A hierarchy rule associated to the ETL is automatically created by TrueSight Capacity Optimization to process the Object-Relationship data.