Moviri - Entuity REST Extractor

“Moviri Integrator for BMC Helix Continuous Optimization – Entuity_Rest” allows extracting data from the Entuity network monitoring tool into BMC Helix Continuous Optimization for relevant capacity metrics.

The integration supports the extraction of both performance and configuration data across network components monitored by Entuity. Furthermore the connector is able to replicate relationships and logical dependencies among entities defined in Entuity into BMC Helix Continuous Optimization, thus enabling grouping and multiple perspectives.

The connector supports configuration parameters that allow devices filtering, metrics filtering, limiting data volume processed in a single run and many other settings.


The documentation is targeted at BMC Helix Continuous Optimization administrators, in charge of configuring and monitoring the integration between BMC Helix Continuous Optimization and Entuity.


Requirements

Supported versions of data source software

  • Entuity v20
  • Entuity v21

Supported configurations of data source software

Moviri - Entuity REST Extractor requires the creation of two Entuity Data Access templates provided in this document.

Installation

The latest version of the extractor is available at Maintenance > Additional Packages > Moviri - Entuity REST extractor.

Creating custom metrics (TODO)


NETIF_*_NUM metrics count physical interfaces only. In order to get the comprehensive count of physical and virtual interfaces, three custom metrics are used:

  • NETIF_PV_UP_NUM_C
  • NETIF_PV_DOWN_NUM_C
  • NETIF_PV_OTHER_NUM_C

In BMC Helix Continuous Optimization, custom metrics must be manually created:

  1. Open the Administration page and click DATA WAREHOUSE > Dataset & Metrics > Add metric.
  2. Use the following configurations for each of the metrics:


Name

Label

Description

Valtype

Dataset

NETIF_PV_DOWN_NUM_C

Physical and virtual interfaces DOWN number

Count of interfaces (physical+virtual) with operational status DOWN

Generic counter, absolute value

[5] SYSNET

NETIF_PV_OTHER_NUM_C

Physical and virtual interfaces not UP or DOWN num

Count of interfaces (physical+virtual) with operational status OTHER

Generic counter, absolute value

[5] SYSNET

NETIF_PV_UP_NUM_C

Physical and virtual interfaces UP number

Count of interfaces (physical+virtual) with operational status UP

Generic counter, absolute value

[5] SYSNET



Data source check and configuration

Preparing to connect to the data source software

In order to extract configuration and performance metrics, the extractor makes use of Entuity API Data Access Interface which is based on templates. It is therefore necessary to create the required templates on your Entuity environment.

Necessary XML definitions of the templates are provided in this guide:

  • BHCO_EntuityRESTextractor_DataAccessTemplate_Conf.xml
  • BHCO_EntuityRESTextractor_DataAccessTemplate_Perf.xml

Please note that template names can be changed, but ETL connection properties must reflect the changes. See ‘Connection parameters configuration’ chapters for details.

Please refer to the official documentation for further information: https://support.entuity.com/hc/en-us/articles/360006535413-RESTful-API


Connector configuration

Connection parameters configuration

The Entuity REST ETL needs to be configured in order to successfully connect to Entuity REST API. The ETL supports the use of a proxy if needed, and both authentication methods: Basic (user/password) or Token based (manually created access key).

Property Name

Value Type

Required?

Default

Description

Base URL

String

yes

-

Entuity RESTful API base URL (https://<baseurl>, without ending slash ‘/’)

Authentication method

String

yes

-

The chosen authentication method (Basic or Token)

API Auth Token

String

yes (if Token)

-

Entuity access key, if selected authentication method is ‘Token’

API username

String

yes (if Basic)

-

Entuity API username, if selected authentication method is ‘Basic’

API password

String

yes (if basic)

-

Entuity API password, if selected authentication method is ‘Basic’

Use proxy

Boolean

yes

false

To enable proxy settings

Proxy server host

String

no

-

Hostname of the proxy server

Proxy server port

String

no

-

Port of the proxy server

Proxy username

String

no

-

Proxy username, if needed

Proxy password

String

no 

-

Proxy password, if needed

Entuity Data Access template for Conf Metrics

String

yes

BHCO_conf

Name of the data access template used for conf metrics. Default value is the name used in the xml definition provided.

Entuity Data Access template for Perf Metrics

String

yes

BHCO_perf

Name of the data access template used for perfmetrics. Default value is the name used in the xml definition provided.


Connector configuration attributes


Property Name

Value Type

Required?

Default

Description

Time management

Last counter

String

yes (first run)

-

Starting timestamp to extract data from for the first run of the ETL, in UTC TZ [YYYY-MM-DD HH24:00:00]. If a last counter is available from a previous run, then this will be ignored.

Maximum hours to extract

Number

yes

24

Limit max number of hours extracted in a single run of the ETL. It helps preventing overloading of the data source and extractor in historical recovery phases. In a steady state situation it is recommended to be greater than the scheduling interval minus lag.

Lag hours

Number

yes

1

Lag number of hours. ETL will stop at the current time minus the lag period if needed to prevent partial extraction of data not yet available.

Entity relationships and naming

Ignore Entuity views relationships and attach all the devices to root domain

Boolean

yes

false

Attach all systems to ETL root domain, by default the ETL extracts the Entuity relationships information

Use port alias for interfaces

Boolean

yes

false

Use port alias as (subobj) name for ports. Fallback order for the attributes to be used as subobjnm is: portAlias > portDescr > portName > ifDescr.

Fallback value is used if the higher priority attribute has one of the following values: null, ??, n/a

Filters on Entuity servers/devices to import

Enable regex for views white and black lists

Boolean

yes

false

Enable regular expressions (‘;’ separator) to be used in views white and black lists

Views white list

String

no

-

Semicolon separated list of Entuity views to be imported. Empty list means all.

Views black list

String

no

-

Semicolon separated list of Entuity views to not import. Empty list means none.

Enable regex for devices white and black lists

Boolean

yes

false

Enable regular expressions (‘;’ separator) to be used in devices white and black lists

Devices white list

String

no

-

Semicolon separated list of Entuity devices/servers to be imported. Empty list means all.

Devices black list

String

no

-

Semicolon separated list of Entuity devices/servers to not import. Empty list means none.

Enable regex for interfaces white and black lists

Boolean

yes

false

Enable regular expressions (‘;’ separator) to be used in interfaces white and black lists

Interfaces white list

String

no

-

Semicolon separated list of Entuity interfaces (devicename:interface) to be imported. Empty list means all.

Interfaces black list

String

no

-

Semicolon separated list of Entuity interfaces (devicename:interface) to not import. Empty list means none.

Enable regex for QoS classes white and black lists

Boolean

yes

false

Enable regular expressions (‘;’ separator) to be used in QoS classes white and black lists

QoS classes white list

String

no

-

Semicolon separated list of Entuity QoS classes to be imported. Empty list means all.

QoS classes black list

String

no

-

Semicolon separated list of Entuity QoS classes to not import. Empty list means none.

Filters on metrics to import

Import virtual interfaces perf metrics

Boolean

yes

false

Collect performance metrics of virtual interfaces

Import loopbakc interfaces perf metrics

Boolean

yes

false

Collect performance metrics of loopback interfaces

Import devices system metrics

Boolean

yes

false

Collect performance CPU and memory metrics of devices

Import devices SNMP response time and ICMP latency

Boolean

yes

false

Collect devices SNMP response time and CIMP latency

Import interfaces traffic metrics

Boolean

yes

true

Collect interfaces traffic metrics (rate and util%)

Import interfaces error metrics

Boolean

yes

true

Collect interfaces error metrics (fault and discards)

Import interfaces QoS metrics

Boolean

yes

false

Collect QoS classes metrics (bitrate and drops)

Import firewall connections metric

Boolean

yes

false

Collect firewall connections count metric

Import active sessions metric

Boolean

yes

false

Collect active sessions count metric

List of BHCO metrics to import

String

no

-

Semicolon separated  list of BHCO metrics for fine-grain metric filtering. If empty collects all. If not, empty collect specified metrics only.

Filters on device types to import

Import ‘Managed Host’ devices

Boolean

yes

false

Import entities of type “Managed Host”

Import ‘Virtualization Platform’ devices

Boolean

yes

false

Import entities of type “Virtualization Platform”

Historical data extraction

In order to perform an historical recovery for the first run of the ETL, set the “Last Counter” property of the run configuration with the desired TS to start the extraction from. After the first run, this value will be ignored, and the stored updated last counter will be used instead.

Based on the amount of data to be retrieved, evaluate to split the historical recovery into multiple runs of the ETL. Use “Maximum hours to extract” parameters to limit the number of hours extracted in a single run and schedule the task to be executed with an higher frequency. It helps preventing overloading of the data source and extractor in historical recovery phases.

Once reached the latest data available, schedule the task for the steady state execution (recommended daily) and adjust “Maximum hours to extract” and “Lag hours” accordingly.

Be aware the ETL expects to extract detail data to be aggregated at hourly resolution computing and storing  statistics (avg, min, max).

Depending on the retention policies defined on your Entuity environment, data retrieved to far in the past may be available at lower resolution or may be not available at all, resulting in samples loss and wrong computation of the statistics. Therefore, choose the amount of historical data to be retrieved accordingly to the available retention.

Based on Entuity default retention periods, with a last counter older than 32 days most of the extracted metrics will have daily resolution only or will be unavailable at all

Troubleshooting

For general administration, refer to Working-with-ETLs. 

Configuration and Performance Metrics Mapping

The following table contains the list of supported Entuity object types (“typeName” attribute of the objects) and their mapping to BHCO system types.

Entuity Object Type

BHCO System Type

CUCMManagedHost

Generic

ManagedHost

Generic

ServerDevice

Generic

FirewallDevice

Network - Firewall

RouterDevice

Network - Router

SwitchDevice

Network - Switch

UCSManagerDevice

Network - Switch

LBDevice

Network - Load Balancer

VpnDevice

Network - Device

VirtualizationPlatformDevice

Virtual Machine


Entity Types

The following tables contains the list of supported Entuity object type and their mapping to System Type

Entuity Object Type

System Type

Router

Network - Router

Firewall

Network - Firewall

Load Balancer

Network - Load Balancer

Ethernet Switch

Network - Switch

Token Ring Switch

Network - Switch

ATM Switch

Network - Switch

Matrix Switch

Network - Switch

Managed Host

Generic

VM Platform

Virtual Machine


Performance metrics


Entuity object type

Entuity stream

Entuity metric

CO dataset

CO metric

Scaling factor

PortEx

portTrafficRateChain

inTrafficRate/inSpeed

SYSNET

BYIF_IN_UTIL (BYVIF_* if virtual)

1

PortEx

portTrafficRateChain

outTrafficRate/outSpeed

SYSNET

BYIF_OUT_UTIL (BYVIF_* if virtual)

1

PortEx

portTrafficRateChain

inTrafficRate

SYSNET

BYIF_IN_BIT_RATE (BYVIF_* if virtual)

1

PortEx

portTrafficRateChain

outTrafficRate

SYSNET

BYIF_OUT_BIT_RATE (BYVIF_* if virtual)

1

PortEx

portFaultChain

inDiscards

SYSNET

BYIF_IN_PKT_DISC_PCT

0.01

PortEx

portFaultChain

inFault

SYSNET

BYIF_IN_PKT_ERR_PCT

0.01

PortEx

portFaultChain

outDiscards

SYSNET

BYIF_OUT_PKT_DISC_PCT

0.01

PortEx

portFaultChain

outFault

SYSNET

BYIF_OUT_PKT_ERR_PCT

0.01

PortEx

portFaultChain

greatest(inFault,outFault,inDiscards,outDiscards)

SYSNET

BYIF_PKT_ERR_PCT

0.01

ClassMap

classMapStream

classMapPrePolicyBitRate

SYSNET

BYIFCLASS_PREPOLICY_BIT_RATE

1

ClassMap

classMapStream

classMapPostPolicyBitRate

SYSNET

BYIFCLASS_POSTPOLICY_BIT_RATE

1

ClassMap

classMapStream

classMapDropPkt64/ classMapPrePolicyPkt64

SYSNET

BYIFCLASS_DROP_PCT

1

ClassMap

classMapStream

classMapDropPkt64Rate*3600

SYSNET

BYIFCLASS_DROP_PKT

1

ClassMap

classMapStream

classMapDropPkt64Rate

SYSNET

BYIFCLASS_DROP_PKT_RATE

1

ClassMap

classMapStream

classMapNoBufDropPkt64/classMapPrePolicyPkt64

SYSNET

BYIFCLASS_DROP_NOBUFFER_PCT

1

ProcessorEx

processorUtilChain

CPUUtil

SYSCPU

BYCPU_CPU_UTIL

0.01

memory

memoryStatusChain

memoryUsed

NETDEV

MEM_IOALLOCATED_USED (if memoryName=I/O)

MEM_CPUALLOCATED_USED (if memoryName=PRCESSOR)

1

memory

memoryStatusChain

memoryFree

NETDEV

MEM_IOALLOCATED_FREE (if memoryName=I/O)

MEM_CPUALLOCATED_FREE (if memoryName=PRCESSOR)

1

memory

memoryStatusChain

MemoryFreeContig

NETDEV

MEM_IOALLOCATED_LARGEST_FREE (if memoryName=I/O)

MEM_CPUALLOCATED_LARGEST_FREE (if memoryName=PRCESSOR)

1

MonitoredDevice

ICMPLatency5MinuteRollup

averageLatency5Minute

NETDEV

ICMP_LATENCY

1

DeviceEx

unifiedDeviceCPU_chain

averageDeviceProcessorUtilPercent

SYSGLB

CPU_UTIL

0.01

DeviceEx

unifiedDeviceMemory_chain

averageDeviceMemoryUsedPercent

SYSGLB

MEM_UTIL

0.01

DeviceEx

SNMPResponseTimeChain

snmpResponseTimeAverage

NETDEV

SNMP_RESPONSE_TIME

1

DeviceEx

SNMPResponseTimeChain

snmpFailureRate

NETDEV

SNMP_FAILURE_RATE

1

DeviceEx

SwitchBusUtilClientChain

systemBusUtil

NETDEV

BACKPLANE_UTIL

0.01

DeviceEx

FirewallPerfChain

firewallConnections

NETDEV

NET_CONNECTIONS_CURRENT

1

DeviceEx

HRDeviceStreamChain

ActiveSessions

NETDEV

NET_SESSIONS_CURRENT

1

PortEx

portStatusChain

count(operStatus=1) group by ts

SYSNET

NETIF_PV_UP_NUM_C (all)

NETIF_UP_NUM (where portVirtualIndicator=Physical)

1

PortEx

portStatusChain

count(operStatus=2) group by ts

SYSNET

NETIF_PV_DOWN_NUM_C (all)

NETIF_DOWN_NUM (where portVirtualIndicator=Physical)

1

PortEx

portStatusChain

count(operStatus not in (1,2)) group by ts

SYSNET

NETIF_PV_OTHER_NUM (all)

NETIF_OTHER_NUM (where portVirtualIndicator=Physical)

1


Configuration metrics


Entuity object type

Entuity metric

CO dataset

CO metric

DeviceEx

sysLocation

SYSCNF

LOCATION_SITE

DeviceEx

sysDescr

SYSCNF

DESCRIPTION

DeviceEx

devicePowerConsumption

SYSPWR

PER_CONSUMPTION

DeviceEx

devType

NETDEV

NETDEV_CATEGORY

DeviceEx

devManufacturer

SYSCNF

HW_VENDOR

DeviceEx

devModel

SYSCNF

HW_MODEL

DeviceEx

devVersion

SYSCNF

SW_VER

DeviceEx

devPolledIpAddr

SYSCNF

NET_IP_ADDRESS

DeviceEx

name

SYSCNF

HOST_NAME

memory

memorySize

SYSCNF

TOTAL_REAL_MEM

PortEx

count(portVirtualIndicator=Physical)

SYSCNF

NETIF_NUM

PortEx

count(portVirtualIndicator=Virtual=

SYSNET

NETVIF_NUM

module

moduleName

NETDEV

BYSLOT_MODULE_NAME

module

moduleDescr

NETDEV

BYSLOT_MODULE_DESCR

PortEx

portVirtualIndicator

SYSNET

BYIF_VIRTUAL_INDICATOR

PortEx

portAdminStatus

SYSNET

BYIF_ADMIN_STATUS

PortEx

portDuplex

SYSNET

BYIF_DUPLEX_STATUS

PortEx

portSpare

SYSNET

BYIF_SPARE_STATUS

PortEx

portOperationalStatus

SYSNET

BYIF_STATUS

PortEx

ifDescr

SYSNET

BYIF_DESC

PortEx

max(portInSpeed,portOutSpeed)

SYSNET

BYIF_BANDWITH

PolicyMap

policyMapDirection

SYSNET

BYIFCLASS_POLICYMAP_DIRECTION

ClassMap

classMapBandwidth

SYSNET

BYIFCLASS_BANDWIDTH

PortEx

<port subobjnm>

SYSNET

BYIFCLASS_IF_NAME

ClassMap

classMapName

SYSNET

BYIFCLASS_CLASS_NAME

ClassMap

classMapExtendedName

SYSNET

BYIFCLASS_CLASS_EXTENDED_NAME

ClassMap

classMapDesc

SYSNET

BYIFCLASS_DESCRIPTION


Lookup Fields

The ETL for Entuity supports the multiple lookup methodology available in BMC Helix Continuous Optimization. This means that, for each entity, more than one lookup value is stored.

Lookup values for Systems

Lookup field

Lookup value

ENTUITY_ID

  • Value: <serverId>-<id>
  • Description: where ‘serverId’ is the remote server ‘serverId’ attribute and ‘id’ is the object ‘id’ attribute
  • Example: 5ba829fa-86d9-4246-b4ce-24cc896180c0-866

HOSTNAME

  • Value: <displayName>
  • Description: where ‘displayName’ is the object ‘displayName’ attribute
  • Example: ca01

_COMPATIBILITY_

  • Value: <displayName>
  • Description: where ‘displayName’ is the object ‘displayName’ attribute
  • Example: ca01


Strong lookup fields: HOSTNAME

Weak lookup fields: ENTUITY_ID##_COMPATIBILITY_

Lookup values for Domains

Lookup field

Lookup value

ENTUITY_ID

  • Value: <viewFullName>
  • Description: where ‘viewFullName’ is the view ‘viewFullName’ attribute
  • Example: My Network/All Objects

HOSTNAME

  • Value: <viewFullName>
  • Description: where ‘viewFullName’ is the view ‘viewFullName’ attribute
  • Example: My Network/All Objects

_COMPATIBILITY_

  • Value: <viewFullName>
  • Description: where ‘viewFullName’ is the view ‘viewFullName’ attribute
  • Example: My Network/All Objects


Strong lookup fields: HOSTNAME

Weak lookup fields: ENTUITY_ID##_COMPATIBILITY_


                 

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*