Service Engine

Description

A service engine is a specialized, lightweight deployment of the BMC Helix Network Management virtual appliance (on its own separate server) that is configured to run only specific BMC Helix Network Management data collection and processing services.

The use of service engines are required for BMC Helix Network Management SaaS deployments, where they provide core data collection and processing services within a customer's secured network environment.

For on-premise BMC Helix Network Management deployments, they are considered optional (but highly recommended) performance additions; as they offload the work of performing resource-intensive services from the BMC Helix Network Management server to the service engine. This is of value because the use of a service engine to collect and process data for traditionally high volume tasks can dramatically reduce the processing load of the server on which BMC Helix Network Management is deployed.

Service engines communicate with your BMC Helix Network Management deployment on port TCP/443, so that port must be open and available on both the service engine appliance and (for on-premise deployments) your BMC Helix Network Management primary appliance.

The following services are available for activation on a service engine (see below for more information on each service):

BMC Helix Network Management Remote Poller - Device availability monitoring and performance data collection and processing (on-premise deployments only).
BMC Helix Network Management Remote Collector - Device availability monitoring and performance data collection and processing.
BMC Helix Network Management Log Collector - Log data collection and processing.
BMC Helix Network Management Traffic Collector - Network application traffic flow data collection and processing.

Any or all of these services may be run on a single service engine in any combination (except remote poller which should always be run by itself). As your network grows, you can deploy any number of service engines using additional separate servers. Service engines are licensed individually as add-ons to your BMC Helix Network Management deployment.

Remote Collector or Remote Poller?

The difference between collectors and pollers is the direction of communication. Collectors collect data within their local network and initiate a connection outbound from their location to the BMC Helix Network Management server location, making them ideal for deployment behind a security perimeter where only outbound communication is allowed. While pollers also collect data within their local network, they must be able to receive inbound communication from the BMC Helix Network Management server and communicate back, as BMC Helix Network Management initiates the communication with the service engine to retrieve its collected data. This means bi-directional communication is required for a poller to work.

Deployed service engines running the remote poller service (only) may optionally be organized into service engine groups for load balancing and redundancy purposes. See Service Engine Group for more information.

Details

When a service engine is deployed for use in BMC Helix Network Management, it consists of two parts:

The actual service engine virtual appliance. This is the virtual appliance deployed within the network you wish to monitor. It connects to your BMC Helix Network Management deployment and is associated with the managed device below.
A managed device representing the service engine within BMC Helix Network Management. This is used used for monitoring the service engine and configuring its settings. It is treated as any other managed device within BMC Helix Network Management, which includes having its own Device Dashboard.

All data related to a particular service is collected and processed on the service engine appliance instead of by BMC Helix Network Management itself. BMC Helix Network Management then queries the connected service engine (approximately every 1 minute) through a RESTful API to retrieve the processed data for display in its own UI. The connection between BMC Helix Network Management and a service engine is on-demand, from BMC Helix Network Management to the service engine, using a secure question and answer framework.

In the event that a BMC Helix Network Management on-premises deployment loses communication with any of its connected service engines, each service engine appliance will still continue to collect and process data for as long as it can still communicate with the devices it monitors. Each service engine caches its collected data for 30 minutes. Once communication with its service engines is restored, BMC Helix Network Management is updated with the cached data. Note that this applies to BMC Helix Network Management on-premises deployments only. Service engines for BMC Helix Network Management SaaS deployments do not have this caching ability, and permanent gaps in historical data will be present for the time during which a service engine is unable to communicate with a BMC Helix Network Management SaaS deployment.

In the event that a service engine itself goes down (server failure, etc.), its host availability service check will fail and generate an alarm (same as any other managed device), opening an incident. Additionally, the device polling status service checks of all devices monitored by that service engine will also automatically fail. The alarms for those failing checks will then be correlated as related alarms in the service engine host down incident.

If you would like to stop using a service engine for a particular service, remove that service type from all connected service engines. BMC Helix Network Management will automatically resume providing that service itself.

Service engine time synchronization

If your service engines do not have their server time synchronized with your BMC Helix Network Management deployment, the dashboards and reports populated with data from those service engines may show inaccurate timestamps. The best way to avoid this is to make sure that your service engines are set to the correct time zone and have proper NTP configurations.

BMC Helix Network Management Remote Poller Service

(BMC Helix Network Management on-premise deployments only)

This service collects and processes device performance statistics and monitors host and service availability.

In an on-premise deployment without service engines, the BMC Helix Network Management appliance itself performs the work of collecting/processing performance data and host and service availability, as well as all other functions, so large numbers of devices may impact the performance of the BMC Helix Network Management server. A service engine running the remote poller service is intended to reduce the load on the server where the BMC Helix Network Management virtual appliance is installed in on-premise BMC Helix Network Management deployments that monitor large numbers of devices.

See Data Retention for information on how much and long BMC Helix Network Management stores performance data.

BMC Helix Network Management Remote Collector Service

(Required in a BMC Helix Network Management SaaS deployment)

This service collects and processes device performance statistics and monitors host and service availability for devices isolated from BMC Helix Network Management by a firewall.

The service engine running the remote collector service is intended to be deployed inside your organization's security perimeter, where BMC Helix Network Management typically cannot initiate a connection. In this case, the connection is initiated outbound by the service engine, which "calls home" to update your BMC Helix Network Management deployment with its collected data.

For on-premise deployments, a remote collector is ideal for collecting performance and availability data from network devices isolated from the core BMC Helix Network Management appliance by a security perimeter.

For SaaS deployments, since the core BMC Helix Network Management appliance is hosted in a cloud environment, it is almost certain that all network devices to be monitored will be behind a security perimeter, thus a remote collector is required.

See Data Retention for information on how much and long BMC Helix Network Management stores performance data.

BMC Helix Network Management Log Collector Service

This service collects and processes log data, including Windows event logs, syslogs and SNMP traps.

In an on-premise deployment, this service functions similarly to a remote poller, in that it is intended to reduce the load on the server where the BMC Helix Network Management virtual appliance is installed by offloading the work of log collection and processing to the service engine.

When using this service, it is important to note that individual managed devices cannot be configured to send traps/logs to multiple service engines. All traps/logs for a given device must be sent to the same service engine. Failure to do so may result in improper collection of traps/logs for that device. (In the case of service engine groups, all traps/logs must be sent to the same group.)

See Data Retention for information on how much and long BMC Helix Network Management stores log data.

Important

When configuring a service engine to run the log collector service, the primary BMC Helix Network Management appliance stops accepting log data. You must update all relevant devices to send logs directly to the assigned service engine.

BMC Helix Network Management Traffic Collector Service

This service collects and processes network application traffic flow data, including NetFlow, IPFIX and sFlow.

See Data Retention for information on how much and long BMC Helix Network Management stores traffic flow data.

See NetFlow Monitoring for more information about monitoring traffic flow with BMC Helix Network Management.

Important

When you configure a service engine to run the traffic collector service, the primary BMC Helix Network Management appliance stops accepting traffic flow data. Make sure all flow sources send data directly to the configured service engine.