Collect network device data from BMC Helix Network Management

Overview of BHNM data collection

BMC Helix Network Management (BHNM, formerly Netreo) collects and monitors routers, switches, firewalls, load balancers, and network devices. To make this data available in BMC Helix Continuous Optimization (BHCO), you must collect and import BHNM output by using a Remote ETL Engine (REE).

For details on installing REE, refer to Installing the Remote ETL Engine - BMC Documentation.

The REE executes the netreo-site-connector.sh script, which connects to BHNM and generates CSV files for object relationships, tags, configuration, and performance metrics.

The files are stored under $BCO_HOME/etl/scripts/netreo/ in the following directories.

relations-etl
tags-etl
config-etl
metrics-etl

BHNM ETLs and directory mapping

ETLs are connector modules that extract BHNM relationship, tag, configuration, and performance data and load each data type into BMC Helix Continuous Optimization for complete analysis.

To import the CSV files generated by the netreo-site-connector.sh script, create the following Generic ETLs. Each ETL reads CSV files from a specific directory under $BCO_HOME/etl/scripts/netreo. The ETL that imports configuration metrics must read from $BCO_HOME/etl/scripts/netreo/config-etl.

ETL types and directory mapping

ETL Type	Purpose	Directory
Generic – Object Relationship Extractor	Import object relationships	relations-etl
Generic – Tag CSV Parser	Import tag data	tags-etl
Generic – CSV Parser	Import configuration metrics	config-etl
Generic – CSV Parser	Import performance metrics	metrics-etl

These ETLs enable BMC Helix Continuous Optimization to process BMC Helix Network Device data for unified capacity analysis.

Benefits of importing BHNM data

Analyze network performance and use it alongside compute and storage data.
Identify network trends and understand their impact on application and infrastructure behavior.

Before you begin

Before you start collecting and importing BHNM metrics, make sure that the Remote ETL Engine (REE) meets the following prerequisites.:

REE requirements

The Remote ETL Engine must have the following installed:

bash
curl
jq (installed locally if not present)

Scripts and usage

The following scripts and files must be available under $BCO_HOME/etl/scripts/netreo/:. These scripts provide the logic for data collection, device type mapping, metric conversion, and dependency checks required for importing BHNM metrics into BMC Helix Continuous Optimization.

Script	Purpose	When to use	How to run
netreo-site-connector.sh	Collects data from BHNM for a specific site and an optional list of device names.	Always as the first step	./netreo-site-connector.sh -e <encoded_password_file> -s <site_name>
supported_net_devices.json	Provides a list of supported out-of-the-box (out-of-the-box) network device types.	Auto-included	No manual run required
include/device_type_mapping.sh	Maps BHNM device types to BMC Helix Continuous Optimization device types and performs metric unit conversions.	Auto-included	No manual run required
include/device_type_mapping.sh	Maps BHNM metrics to BMC Helix Continuous Optimization metrics.	Auto-included	No manual run required
include/tools.sh	Checks for jq and installs it locally if not found.	Auto-included	No manual run required

To create the password file

Before collecting BMC Helix Network Management data, you must store the password in a secure location in a text file that is accessible only to the REE installation owner (for example, cpit). Because this file contains sensitive information, it must be hidden and have restricted permissions to prevent exposure during the integration process.

On the REE host, create a hidden text file to store the BHNM password: touch -f $HOME/.netreo.txt.
Restrict permissions so that only the file owner can read and write the file: chmod 600 $HOME/.netreo.txt
Open the file in a text editor, add the BHNM password as a single line, without adding a newline at the end, and save the file. (For example: vi $HOME/.netreo.txt)
Encode the contents of the password file by running: cat $HOME/.netreo.txt | base64 > $HOME/.neteo.txt
Verify that the file is still owned by the REE installation owner (for example, cpit) and remains in a secure location on the REE host.
Use the encoded password file when running the netreo-site-connector.sh script by specifying it in the password parameter, for example: ./netreo-site-connector.sh -e $HOME/.netreo.txt -s <site_name>

Important:

Use the -e flag to indicate that the password stored in the file is base64-encoded.
During execution, the script decodes the password in memory and passes it through a pipe to curl, so the password does not appear in process lists.
Store the encoded password file in a secure location that is accessible only to the REE user.

To create System Task

You must create a system task on the Remote ETL Engine to schedule and run the netreo-site-connector.sh script. This task collects data from BMC Helix Network Management at regular intervals.

Log in to the BMC Helix Portal.
On the home page, select BMC Helix Continuous Optimization.
Navigate to Administration> ETL & System Tasks > System Tasks.
On the System tasks tab, click Add, and then select Add Process Runner task to create a Process Runner task on the Remote ETL Engine

To configure the Process Runner task, refer to the field description table.

Field	Description
Name	The name of the system task or ETL job. Helps to identify the task in the scheduler or task list.
Deescription	An optional field to provide details about the purpose or functionality of the task.
Maximum execution time before warning	The maximum time the task is allowed to run before a warning is triggered. For example: 4 hours
Frequency	Determines how often the task runs. Can be predefined (for example, Each Day) or Custom (specific number of days/hours).
Predefined frequency	When using a predefined option, select from choices such as Each Day or Each Week.
Start timestamp	The exact time the task starts. Includes: - Hour (0–23) - Minute (0–59) - Weekday (if applicable) - Month day (if applicable)
Custom frequency	Defines the interval for a custom schedule. For example: every 1 day.
Custom start timestamp	The date and time when the task should first run. For example: 09/10/2025 08:47.
Rujning on scheduler	Specifies the Remote ETL Engine (REE) or scheduler node where the task will run.

After configuring all the required fields, verify that the task appears in the System tasks list.
Click Save and schedule the task.

For more information, see Maintaining System tasks.

To edit the system task

You may need to edit the system task to update the schedule, modify parameters, or change the run configuration.

Navigate to Administration > ETL & System Tasks.
On the System tasks tab, locate the task that you created for collecting BHNM data.
In the Actions column next to the task, click Edit.
Update the required fields. You can modify details such as task name, frequencystart time, predefined or custom scheduling options, or the scheduler node.
Click Save to apply the changes.

To configure a Run configuration

A run configuration defines the script, working directory, and parameters used by the Process Runner task. You must configure a run configuration before scheduling the task.

Navigate to Administration > ETL & SYSTEM TASKS > System tasks.
In the System tasks table, click the name of the Process Runner task created for data collection.

In the Run configurations section, click Add run configuration.
Add run configuration - page.png

The following table describes the general configuration fields.

Field		Description
General Configuration
Name		Enter a name for the run configuration.
Deploy status		Indicates whether the configuration is in Simulation or Production mode
Description		Optional description of the run configuration
Process runner execution configuration
Process name list		A list of processes separated by a semicolon. For example, process1;process2.
Notify on error		Select Yes or No to choose whether an error notification is sent.
Command for Netreo		Specifies the full path name to the data collection script: For example, /opt/bmc/BCO/etl/scripts/netreo/netreo-site-connector.sh
Number of parameters for Netreo		Total number of arguments the script requires. For example, nine parameters.
Parameter values for Netro
0	-1	Flag used to specify the host name or IP address.
1	10.177.153.177	The BMC Helix Network Management host IP address to connect to.
2	-P	Flag to indicate the password file path.
3	/opt/bmc/BCO/etl/scripts/netreo/epwd.txt	Path to the encoded password file.
4	-e	Indicates that the password file is base64-encoded.
5	-x	Script-specific flag (internal/expert mode flag as shown in demo).
6	1653086909	Value associated with the -x flag (timestamp or token).
7	-s	Flag to specify the site name or site ID.
8	33	The site ID in BMC Helix Network Management for which device data should be collected.

Click Save.

When the run configuration is set up correctly, the script runs with all required parameters at the scheduled interval and collects the data in the form of CSV files, which are stored on the Remote ETL Engine in the designated Netreo directories.

To configure ETL setup

To import the CSV files collected by the netreo-site-connector.sh script, you must configure four ETLs.

For detailed step-by-step instructions on configuring ETLs, see

Each ETL reads from a specific output directory created by the script. For details about these directories, see the corresponding sections in this document.

ETL type	Dataset	File Location	Catalog and Workspace
Generic – Object Relationship Extractor	OBJREL	$BCO_HOME/etl/scripts/netreo/relations-etl	Use the shared catalog and workspace
Generic – Tag CSV Parser	TAGSET	$BCO_HOME/etl/scripts/netreo/tags-etl	Use the shared catalog and workspace
Generic – CSV Parser (Configuration metrics)	SYSDAT	$BCO_HOME/etl/scripts/netreo/metrics-etl	Use the shared catalog and workspace
Generic – CSV Parser (Performance metrics)	SYSDAT	$BCO_HOME/etl/scripts/netreo/metrics-etl	Use the shared catalog and workspace

After creating all four ETLs, create a Task Chain.

Only the Generic – Object Relationship CSV Parser ETL should have the Object relationships set to new or existing Domain. All other ETLs must leave all new entities in Newly Discovered.

To configure Task Chain

You must configure a task chain to ensure that the Process Runner task runs first and that all related BHNM ETLs execute in the correct sequence.

To configure a Task chain, see the Configuring the Task chain - BMC Documentation page.
Add the four Netreo ETLs in the following execution order:
1. Object Relationship Extractor
2. Tag CSV Parser
3. Generic CSV Parser (Configuration Metrics)
4. Generic CSV Parser (Performance Metrics)
The following figure shows the task chain with the Netreo ETLs added in the correct sequence.
Verify that each ETL (relationships, tags, configuration, and performance) runs and collect the data.
Each ETL writes the imported data to the catalog defined in its configuration. When the ETLs run successfully, the data is written to that catalog.

The task chain must run the four ETLs in the required order. The Object Relationship Extractor runs first, followed by the Tag CSV Parser, and then the two Generic CSV Parser ETLs for configuration and performance metrics. Together, these ETLs import network device data from BHNM into BMC Helix Continuous Optimization.
The data collection task runs at the top of every hour. Because the data collection process requires time to generate the CSV files, schedule the task chain to run a few minutes later so that the files are available before the ETLs start.

To view the catalog configuration

You can view the catalog configuration to confirm that all BHNM ETLs use the same shared catalog. This ensures that the imported data is aligned correctly within BMC Helix Continuous Optimization.

All Netreo ETLs must use the same shared catalog so that relationship, tag, configuration, and performance data are stored together. If the ETLs use different catalogs, the data is separated, and BMC Helix Continuous Optimization cannot build complete objects for the Netreo devices.

In the Tasks chain section, locate the ETL you want to review.
Select Netreo – Generic – Object Relationship Extractor.
Click the Edit icon to open the ETL configuration page.
Review the Catalog field to verify whether the ETL uses a shared or private catalog.

All Netreo ETLs must use the same shared catalog. A shared catalog allows the ETLs to store relationship, tag, configuration, and performance data in one location. If an ETL uses a private catalog, its data is isolated and cannot be accessed by the other ETLs.

To verify the data in BMC Helix Continuous Optimization

Navigate to Workspace > Domain, Services & Applications > <domain> > Systems.
To view the imported metrics, click a device name in the Systems table.
This opens the system details page, where you can view the Summary, Metrics, and Hierarchy.

Troubleshooting data

To troubleshoot Netreo data processing issues

For data collection issues, review the logs from the Process Runner task that executes the netreo-site-connector.sh script.
For data import issues, check the logs of each ETL process that handles the CSV files generated by the script.

Best practices

Use a dedicated REE for BHNM data collection to avoid resource contention.
Run the netreo-site-connector.sh script manually after confirming connectivity and CSV generation before scheduling ETLs.
Keep the directory names (relations-etl, tags-etl, config-etl, metrics-etl) unchanged to avoid ETL failures.
Map catalog/workspace only for the Object Relationship ETL; leave others in Newly Discovered. Review Process Runner and ETL logs regularly from the BMC Helix Continuous Optimization UI to detect issues early.
Verify imported data under the correct <domain>. Clean up old CSV files in the $BCO_HOME/etl/scripts/netreo/ directory to maintain performance.