Moviri - HP NNMi Extractor

“Moviri Integrator for BMC Helix Continuous Optimization – HP NNMi” allows extracting capacity relevant measurements from HP NNMi (HP Network Node Manager i) network monitoring tool and feeding them to CO.

The integration supports the extraction of both performance and configuration data across network components monitored by NNMi. Configuration parameters are available to allow device filtering, metrics filtering, historical recovery, limiting data volume processed in a single run, and many other settings.

The documentation is targeted at BMC Helix Continuous Optimization administrators, in charge of configuring and monitoring the integration between BMC Helix Continuous Optimization and HP NNMi.

• Requirements
• Installation
• Datasource Check and Configuration
• Connector configuration
• Troubleshooting
• Configuration and Performance/Business Metrics Mapping
• Lookup Fields
• Object Relationships

Requirements

Official name of data source software

• “HP NNMi” or HP Network Node Manager i software

Supported versions of data source software

• HP NNMi Version 9.10 to 10.x
• HP NNMi Optic SaaS and on-premise

Supported configurations of data source software

Moviri – HP NNMi Extractor supports NNMi mandatorily configured with SPI "performance for metrics".

Installation

Datasource Check and Configuration

HP NNMi 9.10 to 10.x

Enabling iSPI Performance for Metrics CSV Export process

In order to make data available to third party solutions, HP NNMI with SPI “Performance for metrics” provides a data export command, which can export different csv files on local filesystem.
The integration between BMC Helix Continuous Optimization and HP NNMi takes advantage of this method.

NNM iSPI Performance for Metrics contains a component called “Network Performance Server” (NPS). This component is used by other iSPIs as well. The NPS provides both out-of-the-box and highly customizable reporting tools across any iSPI domain. The iSPI Performance for Metrics in particular focuses on fundamental network interface (SNMP MIB-II) and device component performance variables. It continuously collects, stores, and manages performance-related metrics from across your network and includes multiple pre-built reports for exploring that data.

The configureCsvExport.ovpl utility enables you to export the NPS data (live or summarized) to CSV files. After you initiate the data export process, the NPS places the CSV files in the designated directory at regular intervals of time. In order to configure this process, please follow this procedure:

1.     Log on to the NPS system with the administrative or root privileges.

2.     Run the following command to export the live data:

• On Windows

<Install_Dir>\NNMPerformanceSPI\bin\configureCsvExport.ovpl -p <Extension_Pack> -a "LIVE,<Target_Dir>"

• On Linux

/opt/OV/NNMPerformanceSPI/bin/configureCsvExport.ovpl -p <Extension_Pack> -a "LIVE,<Target_Dir>"

3.     Run the following command to export the summarized data:

• On Windows

<Install_Dir>\NNMPerformanceSPI\bin\configureCsvExport.ovpl -p <Extension_Pack> -a "SUMMARY,<Target_Dir>"

• On Linux

/opt/OV/NNMPerformanceSPI/bin/configureCsvExport.ovpl -p <Extension_Pack> -a "SUMMARY,<Target_Dir>"

Replace in the command described before the <Extension_Pack> placeholder with the name of the considered extension pack. Use the following list for the extension packs provided with the NPS and iSPI Performance for Metrics:

• Interface Health: Interface_Health
• Component Health: Component_Health

Logfile of the process can be found in: /var/opt/OV/NNMPerformanceSPI/logs/prspi.log

NNMi Optic

NNMi Optic does not support the data export module available via the “Network Performance Server” (NPS). To export data from NNMi Optic, it will be necessary to export data from the HP Vertica Database supporting HP NNMi Optic. 

• For NNMi Optic SaaS, the data export flow from the Vertica database must be configured by OpenText. Data will be exported into a set of CSV files made available on a S3 bucket
• For NNMi Optic On-Premise, the data export from the Vertica database should be handled by the BHCO administrator

NNMi Optic SaaS

Please contact your organization's OpenText representative to configure the data export to a shared AWS S3 bucket. The following file contains the Vertica queries to be executed to export data from the NNMi Optic SaaS Vertica database. The file contains four queries:

• Component Query - Aggregated is the query to be executed to export Component Level metrics at 1-hour resolution
• Interface Query - Aggregated is the query to be executed to export Interface Level metrics at 1-hour resolution
• Component Query - Raw is the query to be executed to export Component Level metrics at 5 minutes resolution
• Interface Query - Raw is the query to be executed to export Component Level metrics at 5 minutes resolution

It would be required to schedule only one version of the Component Query and one version of the Interface Query, depending on the customer's requirements. For example, to import component and interface data at the 1-hour resolution, it would be required to schedule only the execution of the "Component Query - Aggregated" and the "Interface Query - Aggregated."

To access the CSV files contained in the AWS S3 Bucket, please refer to the official AWS documentation for available solutions (https://aws.amazon.com/blogs/aws/mountpoint-for-amazon-s3-generally-available-and-ready-for-production-workloads/).

NNMi Optic on-premises

When working with NNMi Optic on-premises, the BHCO administrators must schedule the automatic execution of the queries to export data from the Vertica database into CSV file. The attached SSH script can be used to perform this action. To function properly, the script needs the following requirements to be met:

• Connectivity between the Remote ETL Engine and the NNMi Vertica Database 
• Credentials to connect to the NNMi Vertica Database, with read visibility on the NNMi schema
• Access to the "vsql" client on the Remote ETL Engine. Please refer to the official documentation for more information (https://www.vertica.com/docs/9.2.x/HTML/Content/Authoring/ConnectingToVertica/vsql/Install/InstallingTheVsqlClient.htm)

The script supports the following parameter as input:

• -u, to specify the user to connect to the Vertica Database [default, "admin"]
• -w, to specify the password to connect to the Vertica Database (default, "vertica")
• -h, to specify the host to access the Vertica Database (default, "localhost")
• -p, to specify the port to access the Vertica Database (default, "5433")
• -d, to specify the database name of the Vertica Database (default, "public")
• -s, to specify the schema of the Vertica Database (default, "")
• -e, to specify the max number of hours to cover with a single execution (default, "24")
• -m, to specify the aggregation level. Accepted values are "agg" (for 1-hour resolution) and "raw" (for 5-minutes resolution) (default, "agg")
• -f, to specify the folder where the CSV files containing the Interface metrics will be stored by the script (default, "/opt/bmc/BCO/repository/nnmi")
• -g, to specify the folder where the CSV files containing the Component metrics will be stored by the script (default, "/opt/bmc/BCO/repository/nnmi")

The following command is an example of running the attached SSH script

./nnmi_optic_script.sh  -h hotname -p 5433 -d verticadb  -u dbadmin -w vertica -m agg -e 3 -s mf_shared_provider_default)?

The script can be scheduled on the Remote ETL Engine via crontab or a Process Runner system task.

Please consider that, by default, the script and the "Moviri Integration for HP NNMi" do not clean up the file system containing the CSV files. If necessary, please leverage the "File System Cleaner" System Task. 

Connector configuration

Connector configuration attributes

There are three sections (tabs) in the ETL run configuration page where the behaviour of “Moviri Integrator for BMC Helix Continuous Optimization - HP NNMi” can be specified:

• NNMI Files location
• Time Management
• Extraction Filters

NNMI Files location

Time Management

Extraction Filters

Guidelines to choose the right ETL configuration

In order to set the ETL configuration that match your needs, in this paragraph some common use cases are presented.

Scenario 1: “Typical” network capacity analysis and planning

Goal: ensure the right network capacity (traffic and device resource utilization)
Environment size: medium/big
Environment peculiarity: quite steady
Planning time period:  long (6-12 months)
Outputs: Analyses and forecast on network traffic growth and threshold violation date

Scenario 2: “in-depth” network capacity analysis and planning for particular services/applications/devices

Goal: react on short-term growth (traffic and device resource utilization)
Environment size: small
Environment peculiarity:   new service introduction / across a change of application or devices
Planning time period:  short (weeks)
Outputs: Growth trend, short-term forecast, correlation between resource utilization and service business drivers (users, transactions)

Scenario 3: Quality network capacity analysis and planning

Goal: ensure the right network capacity and quality
Environment size: medium
Environment peculiarity: quite steady
Planning time period:  medium-long (3-12 months)
Outputs: Analyses and forecast on network traffic and errors on interfaces, correlation between errors and device usage metrics

The choice of correct configuration brings you benefits also in terms of shorter execution time and saved disk space on database.

Every NNMi ETL instance configuration applies to all devices selected, if there are different scenario needs for different devices, is recommended to create many ETL instances with different configuration which filters on particular devices (e.g. There are both a big and stable environment (scenario 1) and the introduction of a new service (scenario2))

Data volume considerations

The following table is a reference to estimate the data volume produced daily by NNMi  ETL for each device, interface and module based on which metric filter in ETL configuration is active.

As a reference, it has been chosen a device with 24 active interfaces

In a data center with 3 network interfaces for each server and devices with 24 interfaces, the volume of samples collected using System+ traffic filter (as in par 4.1.1 – scenario 1) is approximately :

Historical data extraction

In order to perform a historical recovery of data, please use “recovery mode properties” settings:
Recovery mode active= true and filling the two properties “since date” and “to date” in order to specify the time window. When recovery mode is active, the lastcounter will not be updated and max day to extract will be ignored.

If the ETL is newly created and has no lastcounter defined, the “default lastcounter” is used.

The pre-condition for an historical extraction is the presence of csv files in the selected time period

Supported Platforms and Metrics

For all metrics imported by the ETL and described below in this section, the following apply:

• Supported 

  BMC Helix Continuous Optimization

   Entity Type: Network device 

Troubleshooting

For ETL troubleshooting, please refer to official BMC documentation available here.

Missing data can also represent a common problem; as shown above, in this case a WARNING would be associated to the ETL task. This can be due to a number of scenarios, among which:

• CSV files are not present in selected directory
• CSV file name pattern does not match the file name
• CSV files have been already parsed and a suffix have been appended after parsing the first time
• BMC Helix Continuous Optimization ETL Recovery mode is active, but in the export directory the files have been deleted.
• The filters in BMC Helix Continuous Optimization ETL custom properties configuration are too strict and does not extract any device, check ETL configuration.

Configuration and Performance/Business Metrics Mapping

The ETL parse csv file produced by NNMi +SPI Performance for metrics. 
The metrics supported by the ETL are stored in NNMi Extension packs “Component” and “Interface”.

Performance metrics

Configuration metrics