Moviri - HP NNMi Extractor
“Moviri Integrator for BMC Helix Continuous Optimization – HP NNMi” allows extracting capacity relevant measurements from HP NNMi (HP Network Node Manager i) network monitoring tool and feeding them to CO.
The integration supports the extraction of both performance and configuration data across network components monitored by NNMi. Configuration parameters are available to allow devices filtering, metrics filtering, historical recovery, limiting data volume processed in a single run and many other settings.
The documentation is targeted at BMC Helix Continuous Optimization administrators, in charge of configuring and monitoring the integration between BMC Helix Continuous Optimization and HP NNMi.
Requirements- Datasource Check and Configuration
- Connector configuration
- Troubleshooting
- Configuration and Performance/Business Metrics Mapping
- Lookup Fields
- Object Relationships
Requirements
Official name of data source software
- “HP NNMi” or HP Network Node Manager i software
Supported versions of data source software
- HP NNMi Version 9.10 to 10.x
Supported configurations of data source software
Moviri – HP NNMi Extractor supports NNMi mandatorily configured with SPI "performance for metrics".
Datasource Check and Configuration
Enabling iSPI Performance for Metrics CSV Export process
In order to make data available to third party solutions, HP NNMI with SPI “Performance for metrics” provides a data export command, which can export different csv files on local filesystem.
The integration between BMC Helix Continuous Optimization and HP NNMi takes advantage of this method.
NNM iSPI Performance for Metrics contains a component called “Network Performance Server” (NPS). This component is used by other iSPIs as well. The NPS provides both out-of-the-box and highly customizable reporting tools across any iSPI domain. The iSPI Performance for Metrics in particular focuses on fundamental network interface (SNMP MIB-II) and device component performance variables. It continuously collects, stores, and manages performance-related metrics from across your network and includes multiple pre-built reports for exploring that data.
The configureCsvExport.ovpl utility enables you to export the NPS data (live or summarized) to CSV files. After you initiate the data export process, the NPS places the CSV files in the designated directory at regular intervals of time. In order to configure this process, please follow this procedure:
1. Log on to the NPS system with the administrative or root privileges.
2. Run the following command to export the live data:
- On Windows
<Install_Dir>\NNMPerformanceSPI\bin\configureCsvExport.ovpl -p <Extension_Pack> -a "LIVE,<Target_Dir>"
- On Linux
/opt/OV/NNMPerformanceSPI/bin/configureCsvExport.ovpl -p <Extension_Pack> -a "LIVE,<Target_Dir>"
3. Run the following command to export the summarized data:
- On Windows
<Install_Dir>\NNMPerformanceSPI\bin\configureCsvExport.ovpl -p <Extension_Pack> -a "SUMMARY,<Target_Dir>"
- On Linux
/opt/OV/NNMPerformanceSPI/bin/configureCsvExport.ovpl -p <Extension_Pack> -a "SUMMARY,<Target_Dir>"
Replace in the command described before the <Extension_Pack> placeholder with the name of the considered extension pack. Use the following list for the extension packs provided with the NPS and iSPI Performance for Metrics:
- Interface Health: Interface_Health
- Component Health: Component_Health
Logfile of the process can be found in: /var/opt/OV/NNMPerformanceSPI/logs/prspi.log
Connector configuration
Connector configuration attributes
There are three sections (tabs) in the ETL run configuration page where the behaviour of “Moviri Integrator for BMC Helix Continuous Optimization - HP NNMi” can be specified:
- NNMI Files location
- Time Management
- Extraction Filters
NNMI Files location
Property Name | Value Type | Required? | Default | Description |
File location | String | Yes | LOCAL | Way to obtain NNMi csv files: |
Directory | String | Yes | Source directory in which csv files are generated | |
File name pattern (lastcounter=%LC%) | String | Yes | f_agg_*Metrics_%LC%_001.csv.gz | Filename pattern, |
Parsed files suffix | String | No | Suffix to append to filename after being parsed | |
Delete parsed files (with suffix) | Yes/No | Yes | No | Perform a delete of already parsed files from source directory (must have the suffix defined in the previous property) and must have lastconter field in filename older than the number of days specified by the following property |
Delete parsed files: days back | Numeric | No | 10 | If previous property has value "yes", specifies the number of days back for delete parsed files from source directory |
Max days to extract (0=no limits) | Numeric | No | 0 | Limit maximum number of days to extract in a single run |
Remote host | String | No | Remote hostname (shown if file location is SCP or Windows share) | |
Username | String | No | Username touse for connection (shown if file location is SCP or Windows share) | |
Password | Password | No | Password to use for user defined in the previous property (shown if file location is SCP or Windows share) |
Time Management
Property Name | Value Type | Required? | Default | Description |
Assign TS to timezone | String | String representing the timezone to which samples should be translated (if applicable). The property accept any format for the Java Timezone (e.g. either an abbreviation such as "PST", a full name such as "America/Los_Angeles") | ||
Lastcounter pattern | String | Yes | * | Wildcard expression to identify lastcounter in file name pattern property. For NNMi 9.10 please do not change default value |
Default Lastcounter (YYYY-MM-DD HH24:MI:SS) | String | No | If lastcounter is not already set, use this entry | |
Recovery mode active | Yes/No | Yes | No | When recovery is active, ETL extracts data in the time window specified by the next two properties, lastcounter will not be updated and max days to extract will be ignored |
Since (YYYY-MM-DD HH24:MI:SS) | String | No | “Since” timestamp when recovery mode is active. Date format is specified between the brackets | |
To (YYYY-MM-DD HH24:MI:SS) | String | No | “To” timestamp when recovery mode is active. Date format is specified between the brackets |
Extraction Filters
Guidelines to choose the right ETL configuration
In order to set the ETL configuration that match your needs, in this paragraph some common use cases are presented.
Scenario 1: “Typical” network capacity analysis and planning
Goal: ensure the right network capacity (traffic and device resource utilization)
Environment size: medium/big
Environment peculiarity: quite steady
Planning time period: long (6-12 months)
Outputs: Analyses and forecast on network traffic growth and threshold violation date
Scenario 2: “in-depth” network capacity analysis and planning for particular services/applications/devices
Goal: react on short-term growth (traffic and device resource utilization)
Environment size: small
Environment peculiarity: new service introduction / across a change of application or devices
Planning time period: short (weeks)
Outputs: Growth trend, short-term forecast, correlation between resource utilization and service business drivers (users, transactions)
Scenario 3: Quality network capacity analysis and planning
Goal: ensure the right network capacity and quality
Environment size: medium
Environment peculiarity: quite steady
Planning time period: medium-long (3-12 months)
Outputs: Analyses and forecast on network traffic and errors on interfaces, correlation between errors and device usage metrics
The choice of correct configuration brings you benefits also in terms of shorter execution time and saved disk space on database.
Every NNMi ETL instance configuration applies to all devices selected, if there are different scenario needs for different devices, is recommended to create many ETL instances with different configuration which filters on particular devices (e.g. There are both a big and stable environment (scenario 1) and the introduction of a new service (scenario2))
Data volume considerations
The following table is a reference to estimate the data volume produced daily by NNMi ETL for each device, interface and module based on which metric filter in ETL configuration is active.
Metric type | Component | ETL filters on metric to import | Daily detail samples: 5 min polling (file “live”) | Daily hour samples (file “agg”) |
Performance | Device | System resources utilization metrics | 2300 | 192 |
Interface | Interfaces traffic metrics | 1152 | 96 | |
Interface | Interfaces error metrics (detail) | 1152 | 96 | |
Interface | Interfaces error metrics (basic) | 288 | 24 | |
Configuration | Device | no filter | 1440 | 60 |
Interface | no filter | 1728 | 72 |
As a reference, it has been chosen a device with 24 active interfaces
Device with 24 active interfaces | ||
ETL filters on metric to import | Daily detail samples | Daily hour samples |
System | 45212 | 1980 |
System+traffic | 72860 | 4284 |
System+traffic+errors basic | 79772 | 4380 |
System+traffic+errors detail | 100508 | 6588 |
In a data center with 3 network interfaces for each server and devices with 24 interfaces, the volume of samples collected using System+ traffic filter (as in par 4.1.1 – scenario 1) is approximately :
Server number | Network interfaces needed | Network devices needed | Daily detail samples (Millions) | Daily hour samples (Millions) |
100 | 300 | 12,5 | 0,9 | 0,05 |
1000 | 3000 | 125 | 9,1 | 0,53 |
10000 | 30000 | 1250 | 91,0 | 5,3 |
100000 | 300000 | 12500 | 910,0 | 53,0 |
Historical data extraction
In order to perform a historical recovery of data, please use “recovery mode properties” settings:
Recovery mode active= true and filling the two properties “since date” and “to date” in order to specify the time window. When recovery mode is active, the lastcounter will not be updated and max day to extract will be ignored.
If the ETL is newly created and has no lastcounter defined, the “default lastcounter” is used.
The pre-condition for an historical extraction is the presence of csv files in the selected time period
Supported Platforms and Metrics
For all metrics imported by the ETL and described below in this section, the following apply:
Supported
BMC Helix Continuous Optimization
Entity Type: Network device
Troubleshooting
For ETL troubleshooting, please refer to official BMC documentation available here.
Missing data can also represent a common problem; as shown above, in this case a WARNING would be associated to the ETL task. This can be due to a number of scenarios, among which:
- CSV files are not present in selected directory
- CSV file name pattern does not match the file name
- CSV files have been already parsed and a suffix have been appended after parsing the first time
- BMC Helix Continuous Optimization ETL Recovery mode is active, but in the export directory the files have been deleted.
- The filters in BMC Helix Continuous Optimization ETL custom properties configuration are too strict and does not extract any device, check ETL configuration.
Configuration and Performance/Business Metrics Mapping
The ETL parse csv file produced by NNMi +SPI Performance for metrics.
The metrics supported by the ETL are stored in NNMi Extension packs “Component” and “Interface”.
Performance metrics
File | Component | Column name/ expression | BMC Helix Continuous Optimization | BMC Helix Continuous Optimization Metric | Factor | |
InterfacesMetrics | Interface | Utilization In (physical IF) | SYSNET | BYIF_IN_UTIL | 1 | |
InterfacesMetrics | Interface | Utilization Out (physical IF) | SYSNET | BYIF_OUT_UTIL | 1 | |
InterfacesMetrics | Interface | Utilization In * ifspeed (physical IF) | SYSNET | BYIF_IN_BIT_RATE | 1 | |
InterfacesMetrics | Interface | Utilization Out * ifspeed (physical IF) | SYSNET | BYIF_OUT_BIT_RATE | 1 | |
InterfacesMetrics | Interface | Discards - Packets In / Volume - Packets In (physical IF) | SYSNET | BYIF_IN_PKT_DISC_PCT | 1 | |
InterfacesMetrics | Interface | Errors - Packets In / Volume - Packets In | SYSNET | BYIF_IN_PKT_ERR_PCT | 1 | |
InterfacesMetrics | Interface | Discards - Packets Out / Volume - Packets Out (physical IF) | SYSNET | BYIF_OUT_PKT_DISC_PCT | 1 | |
InterfacesMetrics | Interface | Errors - Packets Out / Volume - Packets Out (physical IF) | SYSNET | BYIF_OUT_PKT_ERR_PCT | 1 | |
InterfacesMetrics | Interface | Errors - Packets / Volume - Packets (physical IF) | SYSNET | BYIF_PKT_ERR_PCT | 1 | |
InterfacesMetrics | Interface | Utilization In (virtual IF) | SYSNET | BYVIF_IN_UTIL | 1 | |
InterfacesMetrics | Interface | Utilization Out (virtual IF) | SYSNET | BYVIF_OUT_UTIL | 1 | |
InterfacesMetrics | Interface | Utilization In * ifspeed (virtual IF) | SYSNET | BYVIF_IN_BIT_RATE | 1 | |
InterfacesMetrics | Interface | Utilization Out * ifspeed (virtual IF) | SYSNET | BYVIF_OUT_BIT_RATE | 1 | |
InterfacesMetrics | Interface | Sum of physical IF in UP status | SYSNET | NETIF_UP_NUM | 1 | |
InterfacesMetrics | Interface | Sum of physical IF in DOWN status | SYSNET | NETIF_DOWN_NUM | 1 | |
ComponentMetrics | CPU | AVG(CPU 5min Utilization / CPU 1min Utilization) | SYSGEN | CPU_UTIL | 1 | |
ComponentMetrics | CPU | CPU 5min Utilization / CPU 1min Utilization | SYSGEN | BYCPU_CPU_UTIL | 1 | |
ComponentMetrics | Processor | Memory Utilization | SYSGEN | MEM_UTIL | 1 | |
ComponentMetrics | I/O | Free Memory | NETDEV | MEM_IOALLOCATED_FREE | 1000 | |
ComponentMetrics | Processor | Free Memory | NETDEV | MEM_CPUALLOCATED_FREE | 1000 | |
ComponentMetrics | I/O | Free Memory* (Memory Utilization /(1-Memory Utilization)) | NETDEV | MEM_IOALLOCATED_USED | 1000 | |
ComponentMetrics | Processor | Free Memory* (Memory Utilization /(1-Memory Utilization)) | NETDEV | MEM_CPUALLOCATED_USED | 1000 | |
ComponentMetrics | System Switching Bus | Backplane Utilization | NETDEV | BACKPLANE_UTIL | 1 |
Configuration metrics
File | Component | NNMi Metric | BMC Helix Continuous Optimization Dataset | BMC Helix Continuous Optimization Metric | |
ComponentMetrics | CPU | Node Location | SYSGLB | LOCATION_SITE | |
ComponentMetrics | CPU | Node Vendor | SYSGLB | HW_VENDOR | |
ComponentMetrics | CPU | Node Family | SYSGLB | HW_MODEL | |
ComponentMetrics | MANAGEMENT_ADDRESS | Component Name | SYSGLB | NET_IP_ADDRESS | |
ComponentMetrics | CPU | Node Name | SYSGLB | HOST_NAME | |
InterfacesMetrics | Interface | Interface type: | SYSNET | BYIF_VIRTUAL_INDICATOR | |
InterfacesMetrics | Interface | Availability | SYSNET | BYIF_STATUS | |
InterfacesMetrics | Interface | count (physical) | SYSNET | NETIF_NUM | |
InterfacesMetrics | Interface | count(virtual) | SYSNET | NETVIF_NUM | |
InterfacesMetrics | Interface | Interface Speed (In:Out) | SYSNET | BYIF_BANDWIDTH | |
InterfacesMetrics | Interface | Interface Alias | SYSNET | BYIF_DESC |
Lookup Fields
Moviri Integrator for BMC Helix Continuous Optimization connector uses standard BMC Helix Continuous Optimization ETL lookup mechanism. One single entry is used as entity identifier: Lookup field = “DEFAULT”.
Object Relationships
Through Object-Relationship panel in the ETL edit run configuration, the ETL arranges the collected systems under a domain of choice.
BMC Helix Continuous Optimization automatically creates a hierarchy rule associated to the ETL to process the Object-Relationship data.