Entuity_Rest
“Moviri Integrator for BMC Helix Continuous Optimization – Entuity_Rest” allows extracting data from the Entuity network monitoring tool into BMC Helix Continuous Optimization for relevant capacity metrics.
The integration supports the extraction of both performance and configuration data across network components monitored by Entuity. Furthermore the connector is able to replicate relationships and logical dependencies among entities defined in Entuity into BMC Helix Continuous Optimization, thus enabling grouping and multiple perspectives.
The connector supports configuration parameters that allow devices filtering, metrics filtering, limiting data volume processed in a single run and many other settings.
The documentation is targeted at BMC Helix Continuous Optimization administrators, in charge of configuring and monitoring the integration between BMC Helix Continuous Optimization and Entuity.
- Requirements
- Installation
- Creating custom metrics (TODO)
- Data source check and configuration
- Connector configuration
- Troubleshooting
- Configuration and Performance Metrics Mapping
- 6.1 Entity Types
- 6.2 Performance metrics
- 6.3 Configuration metrics
- Lookup Fields
Requirements
Supported versions of data source software
- Entuity v20
- Entuity v21
- Entuity v22
Supported configurations of data source software
Moviri - Entuity REST Extractor requires the creation of two Entuity Data Access templates provided in this document.
Installation
The latest version of the extractor is available at Maintenance > Additional Packages > Moviri - Entuity REST extractor.
Creating custom metrics (TODO)
NETIF_*_NUM metrics count physical interfaces only. In order to get the comprehensive count of physical and virtual interfaces, three custom metrics are used:
- NETIF_PV_UP_NUM_C
- NETIF_PV_DOWN_NUM_C
- NETIF_PV_OTHER_NUM_C
In BMC Helix Continuous Optimization, custom metrics must be manually created:
- Open the Administration page and click DATA WAREHOUSE > Dataset & Metrics > Add metric.
- Use the following configurations for each of the metrics:
Name | Label | Description | Valtype | Dataset |
NETIF_PV_DOWN_NUM_C | Physical and virtual interfaces DOWN number | Count of interfaces (physical+virtual) with operational status DOWN | Generic counter, absolute value | [5] SYSNET |
NETIF_PV_OTHER_NUM_C | Physical and virtual interfaces not UP or DOWN num | Count of interfaces (physical+virtual) with operational status OTHER | Generic counter, absolute value | [5] SYSNET |
NETIF_PV_UP_NUM_C | Physical and virtual interfaces UP number | Count of interfaces (physical+virtual) with operational status UP | Generic counter, absolute value | [5] SYSNET |
Data source check and configuration
Preparing to connect to the data source software
In order to extract configuration and performance metrics, the extractor makes use of Entuity API Data Access Interface which is based on templates. It is therefore necessary to create the required templates on your Entuity environment.
Necessary XML definitions of the templates are provided in this guide:
Please note that template names can be changed, but ETL connection properties must reflect the changes. See ‘Connection parameters configuration’ chapters for details.
The two templates can be uploaded to Entuity server through the following curl requests:
curl -k -u "{user}:{password}" -X POST -H "content-type:application/xml" -d {BHCO_conf.xml content} "https://{ENTUITYSERVERDQDN}/api/dataAccessTemplates?media=xml"
curl -k -u "{user}:{password}" -X POST -H "content-type:application/xml" -d {BHCO_perf.xml content} "https://{ENTUITYSERVERDQDN}/api/dataAccessTemplates?media=xml"
Please refer to the official documentation for further information: https://support.entuity.com/hc/en-us/articles/360006535413-RESTful-API
Connector configuration
Connection parameters configuration
The Entuity REST ETL needs to be configured in order to successfully connect to Entuity REST API. The ETL supports the use of a proxy if needed, and both authentication methods: Basic (user/password) or Token based (manually created access key).
Property Name | Value Type | Required? | Default | Description |
Base URL | String | yes | - | Entuity RESTful API base URL (https://<baseurl>, without ending slash ‘/’) |
Authentication method | String | yes | - | The chosen authentication method (Basic or Token) |
API Auth Token | String | yes (if Token) | - | Entuity access key, if selected authentication method is ‘Token’ |
API username | String | yes (if Basic) | - | Entuity API username, if selected authentication method is ‘Basic’ |
API password | String | yes (if basic) | - | Entuity API password, if selected authentication method is ‘Basic’ |
Use proxy | Boolean | yes | false | To enable proxy settings |
Proxy server host | String | no | - | Hostname of the proxy server |
Proxy server port | String | no | - | Port of the proxy server |
Proxy username | String | no | - | Proxy username, if needed |
Proxy password | String | no | - | Proxy password, if needed |
Entuity Data Access template for Conf Metrics | String | yes | BHCO_conf | Name of the data access template used for conf metrics. Default value is the name used in the xml definition provided. |
Entuity Data Access template for Perf Metrics | String | yes | BHCO_perf | Name of the data access template used for perfmetrics. Default value is the name used in the xml definition provided. |
Connector configuration attributes
Property Name | Value Type | Required? | Default | Description |
Time management | ||||
Last counter | String | yes (first run) | - | Starting timestamp to extract data from for the first run of the ETL, in UTC TZ [YYYY-MM-DD HH24:00:00]. If a last counter is available from a previous run, then this will be ignored. |
Maximum hours to extract | Number | yes | 24 | Limit max number of hours extracted in a single run of the ETL. It helps preventing overloading of the data source and extractor in historical recovery phases. In a steady state situation it is recommended to be greater than the scheduling interval minus lag. |
Lag hours | Number | yes | 1 | Lag number of hours. ETL will stop at the current time minus the lag period if needed to prevent partial extraction of data not yet available. |
Entity relationships and naming | ||||
Ignore Entuity views relationships and attach all the devices to root domain | Boolean | yes | false | Attach all systems to ETL root domain, by default the ETL extracts the Entuity relationships information |
Use port alias for interfaces | Boolean | yes | false | Use port alias as (subobj) name for ports. Fallback order for the attributes to be used as subobjnm is: portAlias > portDescr > portName > ifDescr. Fallback value is used if the higher priority attribute has one of the following values: null, ??, n/a |
Filters on Entuity servers/devices to import | ||||
Enable regex for views white and black lists | Boolean | yes | false | Enable regular expressions (‘;’ separator) to be used in views white and black lists |
Views white list | String | no | - | Semicolon separated list of Entuity views to be imported. Empty list means all. |
Views black list | String | no | - | Semicolon separated list of Entuity views to not import. Empty list means none. |
Enable regex for devices white and black lists | Boolean | yes | false | Enable regular expressions (‘;’ separator) to be used in devices white and black lists |
Devices white list | String | no | - | Semicolon separated list of Entuity devices/servers to be imported. Empty list means all. |
Devices black list | String | no | - | Semicolon separated list of Entuity devices/servers to not import. Empty list means none. |
Enable regex for interfaces white and black lists | Boolean | yes | false | Enable regular expressions (‘;’ separator) to be used in interfaces white and black lists |
Interfaces white list | String | no | - | Semicolon separated list of Entuity interfaces (devicename:interface) to be imported. Empty list means all. |
Interfaces black list | String | no | - | Semicolon separated list of Entuity interfaces (devicename:interface) to not import. Empty list means none. |
Enable regex for QoS classes white and black lists | Boolean | yes | false | Enable regular expressions (‘;’ separator) to be used in QoS classes white and black lists |
QoS classes white list | String | no | - | Semicolon separated list of Entuity QoS classes to be imported. Empty list means all. |
QoS classes black list | String | no | - | Semicolon separated list of Entuity QoS classes to not import. Empty list means none. |
Filters on metrics to import | ||||
Import virtual interfaces perf metrics | Boolean | yes | false | Collect performance metrics of virtual interfaces |
Import loopbakc interfaces perf metrics | Boolean | yes | false | Collect performance metrics of loopback interfaces |
Import devices system metrics | Boolean | yes | false | Collect performance CPU and memory metrics of devices |
Import devices SNMP response time and ICMP latency | Boolean | yes | false | Collect devices SNMP response time and CIMP latency |
Import interfaces traffic metrics | Boolean | yes | true | Collect interfaces traffic metrics (rate and util%) |
Import interfaces error metrics | Boolean | yes | true | Collect interfaces error metrics (fault and discards) |
Import interfaces QoS metrics | Boolean | yes | false | Collect QoS classes metrics (bitrate and drops) |
Import firewall connections metric | Boolean | yes | false | Collect firewall connections count metric |
Import active sessions metric | Boolean | yes | false | Collect active sessions count metric |
List of BHCO metrics to import | String | no | - | Semicolon separated list of BHCO metrics for fine-grain metric filtering. If empty collects all. If not, empty collect specified metrics only. |
Filters on device types to import | ||||
Import ‘Managed Host’ devices | Boolean | yes | false | Import entities of type “Managed Host” |
Import ‘Virtualization Platform’ devices | Boolean | yes | false | Import entities of type “Virtualization Platform” |
Historical data extraction
In order to perform an historical recovery for the first run of the ETL, set the “Last Counter” property of the run configuration with the desired TS to start the extraction from. After the first run, this value will be ignored, and the stored updated last counter will be used instead.
Based on the amount of data to be retrieved, evaluate to split the historical recovery into multiple runs of the ETL. Use “Maximum hours to extract” parameters to limit the number of hours extracted in a single run and schedule the task to be executed with an higher frequency. It helps preventing overloading of the data source and extractor in historical recovery phases.
Once reached the latest data available, schedule the task for the steady state execution (recommended daily) and adjust “Maximum hours to extract” and “Lag hours” accordingly.
Be aware the ETL expects to extract detail data to be aggregated at hourly resolution computing and storing statistics (avg, min, max).
Depending on the retention policies defined on your Entuity environment, data retrieved to far in the past may be available at lower resolution or may be not available at all, resulting in samples loss and wrong computation of the statistics. Therefore, choose the amount of historical data to be retrieved accordingly to the available retention.
Based on Entuity default retention periods, with a last counter older than 32 days most of the extracted metrics will have daily resolution only or will be unavailable at all
Troubleshooting
For general administration, refer to Working-with-ETLs.
Configuration and Performance Metrics Mapping
6.1 Entity Types
The following table contains the list of supported Entuity object types (“typeName” attribute of the objects) and their mapping to BHCO system types.
Entuity Object Type | BHCO System Type |
CUCMManagedHost | Generic |
ManagedHost | Generic |
ServerDevice | Generic |
FirewallDevice | Network - Firewall |
RouterDevice | Network - Router |
SwitchDevice | Network - Switch |
UCSManagerDevice | Network - Switch |
LBDevice | Network - Load Balancer |
VpnDevice | Network - Device |
VirtualizationPlatformDevice | Virtual Machine |
6.2 Performance metrics
| Entuity object type | Entuity stream | Entuity metric | CO dataset | CO metric | Scaling factor |
| PortEx | portTrafficRateChain | inTrafficRate/inSpeed | SYSNET | BYIF_IN_UTIL (BYVIF_* if virtual) | 1 |
| PortEx | portTrafficRateChain | outTrafficRate/outSpeed | SYSNET | BYIF_OUT_UTIL (BYVIF_* if virtual) | 1 |
| PortEx | portTrafficRateChain | inTrafficRate | SYSNET | BYIF_IN_BIT_RATE (BYVIF_* if virtual) | 1 |
| PortEx | portTrafficRateChain | outTrafficRate | SYSNET | BYIF_OUT_BIT_RATE (BYVIF_* if virtual) | 1 |
| PortEx | portFaultChain | inDiscards | SYSNET | BYIF_IN_PKT_DISC_PCT | 0.01 |
| PortEx | portFaultChain | inFault | SYSNET | BYIF_IN_PKT_ERR_PCT | 0.01 |
| PortEx | portFaultChain | outDiscards | SYSNET | BYIF_OUT_PKT_DISC_PCT | 0.01 |
| PortEx | portFaultChain | outFault | SYSNET | BYIF_OUT_PKT_ERR_PCT | 0.01 |
| PortEx | portFaultChain | greatest(inFault,outFault,inDiscards,outDiscards) | SYSNET | BYIF_PKT_ERR_PCT | 0.01 |
| ClassMap | classMapStream | classMapPrePolicyBitRate | SYSNET | BYIFCLASS_PREPOLICY_BIT_RATE | 1 |
| ClassMap | classMapStream | classMapPostPolicyBitRate | SYSNET | BYIFCLASS_POSTPOLICY_BIT_RATE | 1 |
| ClassMap | classMapStream | classMapDropPkt64/ classMapPrePolicyPkt64 | SYSNET | BYIFCLASS_DROP_PCT | 1 |
| ClassMap | classMapStream | classMapDropPkt64Rate*3600 | SYSNET | BYIFCLASS_DROP_PKT | 1 |
| ClassMap | classMapStream | classMapDropPkt64Rate | SYSNET | BYIFCLASS_DROP_PKT_RATE | 1 |
| ClassMap | classMapStream | classMapNoBufDropPkt64/classMapPrePolicyPkt64 | SYSNET | BYIFCLASS_DROP_NOBUFFER_PCT | 1 |
| ProcessorEx | processorUtilChain | CPUUtil | SYSCPU | BYCPU_CPU_UTIL | 0.01 |
| memory | memoryStatusChain | memoryUsed | NETDEV | MEM_IOALLOCATED_USED (if memoryName=I/O) MEM_CPUALLOCATED_USED (if memoryName=PRCESSOR) | 1 |
| memory | memoryStatusChain | memoryFree | NETDEV | MEM_IOALLOCATED_FREE (if memoryName=I/O) MEM_CPUALLOCATED_FREE (if memoryName=PRCESSOR) | 1 |
| memory | memoryStatusChain | MemoryFreeContig | NETDEV | MEM_IOALLOCATED_LARGEST_FREE (if memoryName=I/O) MEM_CPUALLOCATED_LARGEST_FREE (if memoryName=PRCESSOR) | 1 |
| MonitoredDevice | ICMPLatency5MinuteRollup | averageLatency5Minute | NETDEV | ICMP_LATENCY | 1 |
| DeviceEx | unifiedDeviceCPU_chain | averageDeviceProcessorUtilPercent | SYSGLB | CPU_UTIL | 0.01 |
| DeviceEx | unifiedDeviceMemory_chain | averageDeviceMemoryUsedPercent | SYSGLB | MEM_UTIL | 0.01 |
| DeviceEx | SNMPResponseTimeChain | snmpResponseTimeAverage | NETDEV | SNMP_RESPONSE_TIME | 1 |
| DeviceEx | SNMPResponseTimeChain | snmpFailureRate | NETDEV | SNMP_FAILURE_RATE | 1 |
| DeviceEx | SwitchBusUtilClientChain | systemBusUtil | NETDEV | BACKPLANE_UTIL | 0.01 |
| DeviceEx | FirewallPerfChain | firewallConnections | NETDEV | NET_CONNECTIONS_CURRENT | 1 |
| DeviceEx | HRDeviceStreamChain | ActiveSessions | NETDEV | NET_SESSIONS_CURRENT | 1 |
| PortEx | portStatusChain | count(operStatus=1) group by ts | SYSNET | NETIF_PV_UP_NUM_C (all) NETIF_UP_NUM (where portVirtualIndicator=Physical) | 1 |
| PortEx | portStatusChain | count(operStatus=2) group by ts | SYSNET | NETIF_PV_DOWN_NUM_C (all) NETIF_DOWN_NUM (where portVirtualIndicator=Physical) | 1 |
| PortEx | portStatusChain | count(operStatus not in (1,2)) group by ts | SYSNET | NETIF_PV_OTHER_NUM (all) NETIF_OTHER_NUM (where portVirtualIndicator=Physical) | 1 |
6.3 Configuration metrics
| Entuity object type | Entuity metric | CO dataset | CO metric |
| DeviceEx | sysLocation | SYSCNF | LOCATION_SITE |
| DeviceEx | sysDescr | SYSCNF | DESCRIPTION |
| DeviceEx | devicePowerConsumption | SYSPWR | PER_CONSUMPTION |
| DeviceEx | devType | NETDEV | NETDEV_CATEGORY |
| DeviceEx | devManufacturer | SYSCNF | HW_VENDOR |
| DeviceEx | devModel | SYSCNF | HW_MODEL |
| DeviceEx | devVersion | SYSCNF | SW_VER |
| DeviceEx | devPolledIpAddr | SYSCNF | NET_IP_ADDRESS |
| DeviceEx | name | SYSCNF | HOST_NAME |
| memory | memorySize | SYSCNF | TOTAL_REAL_MEM |
| PortEx | count(portVirtualIndicator=Physical) | SYSCNF | NETIF_NUM |
| PortEx | count(portVirtualIndicator=Virtual= | SYSNET | NETVIF_NUM |
| module | moduleName | NETDEV | BYSLOT_MODULE_NAME |
| module | moduleDescr | NETDEV | BYSLOT_MODULE_DESCR |
| PortEx | portVirtualIndicator | SYSNET | BYIF_VIRTUAL_INDICATOR |
| PortEx | portAdminStatus | SYSNET | BYIF_ADMIN_STATUS |
| PortEx | portDuplex | SYSNET | BYIF_DUPLEX_STATUS |
| PortEx | portSpare | SYSNET | BYIF_SPARE_STATUS |
| PortEx | portOperationalStatus | SYSNET | BYIF_STATUS |
| PortEx | ifDescr | SYSNET | BYIF_DESC |
| PortEx | max(portInSpeed,portOutSpeed) | SYSNET | BYIF_BANDWITH |
| PolicyMap | policyMapDirection | SYSNET | BYIFCLASS_POLICYMAP_DIRECTION |
| ClassMap | classMapBandwidth | SYSNET | BYIFCLASS_BANDWIDTH |
| PortEx | <port subobjnm> | SYSNET | BYIFCLASS_IF_NAME |
| ClassMap | classMapName | SYSNET | BYIFCLASS_CLASS_NAME |
| ClassMap | classMapExtendedName | SYSNET | BYIFCLASS_CLASS_EXTENDED_NAME |
| ClassMap | classMapDesc | SYSNET | BYIFCLASS_DESCRIPTION |
Lookup Fields
The ETL for Entuity supports the multiple lookup methodology available in BMC Helix Continuous Optimization. This means that, for each entity, more than one lookup value is stored.
Lookup values for Systems
Lookup field | Lookup value |
ENTUITY_ID |
|
HOSTNAME |
|
_COMPATIBILITY_ |
|
Strong lookup fields: HOSTNAME
Weak lookup fields: ENTUITY_ID##_COMPATIBILITY_
Lookup values for Domains
Lookup field | Lookup value |
ENTUITY_ID |
|
HOSTNAME |
|
_COMPATIBILITY_ |
|
Strong lookup fields: HOSTNAME
Weak lookup fields: ENTUITY_ID##_COMPATIBILITY_