Default language.

Important This documentation space contains information about the on-premises version of BMC Helix Discovery. If you are using the SaaS version of BMC Helix Discovery, see BMC Helix Discovery (SaaS).

Consolidation

Consolidation refers to centralizing discovery data on other BMC Discovery appliances or syncing the inferred model to BMC Helix Discovery instances. There are two types of consolidation:

  • Consolidating directly discovered data (DDD) — DDD is sent to receiving BMC Discovery appliances, which then process the data to create their own inferred model. You might want to use consolidation between on-premises appliances where networks are segmented or access is restricted in some other way, such as a short scanning window or company security policies. In many cases, the BMC Discovery Outpost can be used to replace a scanning appliance.
  • Consolidating the inferred model — Where just the inferred model is consolidated, no DDD from discovered targets is transferred to the receiver. Your company security policies might prohibit sensitive data contained in DDD from being stored or processed off-site. Consolidation of the inferred model can be used to synchronize data from existing on-premises BMC Discovery appliances into the BMC Helix ecosystem, where other products, such as BMC Helix AIOPs, can use BMC Discovery's rich data.

Consolidation of the inferred model performs an ongoing synchronization of inferred data from one BMC Discovery system to another. It is not intended for migrating data from BMC Discovery on-premises systems to BMC Helix Discovery SaaS systems before scanning with the SaaS system.

As a general rule with consolidation and with BMC Discovery, we recommend avoiding scanning the same targets from different BMC Discovery Outpost or appliances.

Related topics

Consolidating-directly-discovered-data

Consolidating-the-inferred-model

How-nodes-are-removed

The intended configurations for consolidation are as follows:

  • Consolidation of DDD only.
  • Consolidation of inferred data only.
  • Two-step consolidation where step 1 is DDD and step 2 is inferred.

The following diagrams show configurations for consolidation:

Consolidation configurations.png

Consolidation can also use multiple sources consolidating to a single receiver:

Multiple sources.png

The following screenshot shows the consolidation UI for a BMC Discovery appliance receiving DDD from another BMC Discovery appliance and sending the inferred model to another BMC Discovery appliance.

Consolidation-DDDandInferred.png

Info

It is not expected or recommended to have:

  • A single sender sending DDD and inferred data to two receivers.
  • A single receiver receiving DDD and inferred data from two different senders. 

Consolidation of directly discovered data

Consolidating directly discovered data involves a BMC Discovery appliance sending DDD to a receiving BMC Discovery appliance or appliances, which then process the data to create their own inferred model. You might want to use consolidation between on-premises appliances in the following scenarios:  

  • Firewalled environments — When an environment is divided by firewalls so that a single appliance cannot reach all parts of the network, a scanner can be deployed on each network section blocked by a firewall. The scanners can all send data back to a central consolidator.
  • Restricted (time) scanning windows — Where a discovery window is short, a single appliance might be unable to complete a scan of a large range of IP addresses during the permitted time. Sharing the IP addresses between multiple scanners means each smaller scan can be completed in less time, and the results can be consolidated and viewed on the consolidator. You may consider using a cluster in this situation.
  • Restricted (policy) networks — Certain lines of business might enforce policies on the control or visibility of IT infrastructure in their environments. Scanners can be deployed where such policies limit or prohibit access; the scanners all send data back to a central consolidator.

Multiple scanners can be deployed in each situation, and their data can be consolidated into a central consolidator. The consolidator is then used for reporting and provides a coherent view of the entire scanned network, while each scanner provides a view of the network segment or ranges it scans. A consolidator must be set as one that accepts connections or feeds from scanners. Scanners must, in turn, register with a consolidator. Consolidators can also scan. Any consolidator appliance can also be used to perform discovery in its own right.

Note

Where a scanning appliance is in a scope, that scope information is sent to the consolidator. Scanners and consolidators do not have to be in the same scope.

Consolidation and clusters

For consolidation purposes, clusters operate similarly to stand-alone appliances. Stand-alone senders can consolidate to any member of a cluster. When using a cluster as a sender, you configure consolidation on any member UI, but the cluster coordinator sends information to the receiving cluster. The scanning cluster can consolidate to any member of the receiving cluster. 

Note

DDD consolidation uses port 25032 to communicate. The sender must be able to connect to port 25032 on the receiver. You must configure any firewalls between senders and receivers to allow this traffic. For clusters that act as senders, you must open port 25032 on all members. For clusters that act as receivers, you must open port 25032 on the coordinator, but if you change the cluster coordinator, you must open port 25032 on the new coordinator.

For information about configuring consolidation of DDD, see Consolidating-directly-discovered-data.

Restrictions 

The following restrictions on sending and receiving consolidation data apply to appliances with consolidation configured:

  • DDD cannot be consolidated to a BMC Helix Discovery instance.
  • A DDD receiver cannot also be a DDD sender.
  • A DDD sender can send to one or more inferred senders.

In general, consolidation of DDD works across supported versions of BMC Discovery and it is not mandatory that scanning appliances and consolidating appliances are the same versions. A consolidator can accept data from a scanner of the same or an earlier version, including the level of the service pack. However, in some situations, a consolidator might not get all the expected data from the scanner if the consolidator is running a version with major discovery changes. For example, a 21.3 (12.3) consolidator does not receive some components from an IIS Webserver SI from an earlier scanner. The PowerShell discovery changes in 12.3 changed the DDD sufficiently that the earlier DDD could not be accepted.

The versions are verified when you test the scanner-consolidator connection and when the scanner periodically verifies that the consolidator is still accessible. If you try to consolidate to a consolidating appliance with an earlier version than your scanner, warning messages are shown in the UI.

What data is consolidated in DDD consolidation?

The consolidated data is the BMC Discovery DDD nodes including the data collected by the patterns. The data inferred by the scanners, for example, SoftwareInstance nodes, is not consolidated, but the consolidator infers it again (based on its pattern configuration).

When a host is discovered, and patterns are triggered that run commands on a second host, the DDD for both the hosts is updated. The scanner sends the results of requests on the second host with the consolidation data from the original host. In this way, the commands that are run against another host can be successfully consolidated.

The data imported through CSV in a scanner is not consolidated. It must be imported into all other appliances.

Note

The TKU package version and custom patterns that are loaded on the scanner and consolidators must be the same to infer the same data, for example, SoftwareInstance nodes. Matching the TKU version and custom patterns is not enforced in any way by the system.

Data provenance

Data provenance, or the source of the data, for DDD consolidation in the receiver looks the same as for locally discovered nodes. For example, the host name comes from DeviceInfo method which the receiver ran against the DDD received from the sender. For example:

provenanceDDD.png

Consolidation of the inferred model

Consolidating the inferred model involves a BMC Discovery appliance sending the inferred model to one or more BMC Helix Discovery instances or BMC Discovery appliances. A BMC Helix Discovery instance can send the inferred model to one or more BMC Helix Discovery instances. Consolidating the inferred model is used primarily to synchronize data from existing on-premises BMC Discovery appliances into the BMC Helix ecosystem, where other products, such as BMC Helix AIOPs, can use the rich data discovered. It can also be used to synchronize to another on-premises appliance that might be used, for example, for reporting.

The scope of the inferred data remains the same after consolidation.

For information about configuring consolidation of the inferred model, see Consolidating-the-inferred-model.

What data is consolidated in inferred model consolidation?

All data in the default partition and all changes to that data are sent to the receiver. The data is stored on the receiver in a shadow partition created for that sender. The receiver then unifies the data received in the shadow partition with existing data in the default partition.

No DDD that is retrieved from targets is sent to the receiver. However, the following DDD nodes are sent: 

  • Discovery access node — Contains information about the access to the target, such as the time the target was discovered and whether the discovery was a success or failure. For complete information about the information contained in a discovery access node, see Discovery-Access-node.
  • Discovery run node — Contains information about the scan used to discover the target. Consolidating the inferred model syncs the Company attribute so that multi-tenancy can be supported in CMDB sync from the receiving instance or appliance. For complete information about the information contained in a discovery run node, see Discovery-Run-node.

Provenance information for the inferred data is consolidated so that the UI can display provenance information and can link back to the appliance that holds the DDD containing the information about exactly how the target was discovered.

Inferred data that is consolidated is removed when it is removed from the sender. The two DDD nodes are aged according to DDD aging on the receiver. For more information, see How-nodes-are-removed.

Resynchronization

Consolidation of the inferred model only sends changes, so the model held in the receiver's partition must be precisely synchronized with that in the sender before consolidation can occur. Resynchronization is required when a scanner (sender) connects to a consolidator (receiver) for the first time. Even a new, clean sender that has not performed any discovery is not empty, and its data must be sent to the receiver before consolidation can occur. 

Resynchronization might also be required during normal operation, for example, when you restore a backup on the sender or receiver. Occasionally, you might be informed by the sender that a resynchronization is required.

Restrictions 

The following restrictions on sending and receiving consolidation data apply to appliances or instances with consolidation configured:

  • BMC Helix Discovery (SaaS) instances can only send or receive inferred data.
  • An inferred receiver cannot be used as a sender.
  • A DDD receiver can also send or receive inferred data but cannot do both.

The following restrictions apply to the inferred data that is consolidated:

  • If a pattern on the sender removes an attribute from a node, it is not removed on the receiver, as it is not possible to safely remove attributes that might have come from multiple sources.
  • Where Automatic-grouping is used on the sender, the groups are not sent to the receiver.

Data provenance

For inferred consolidation, the data is discovered by the sender. Consequently, the provenance information at the inferred receiver links back to the sender that performed the discovery. For example, the links, the first one is highlighted in the following figure, link to the same node on the sender:

provenanceInferred.png

The same node on the sender is shown in the following figure:

provenanceInferredSource.png

Service and application models 

Synced service and application models become a new type of model in the receiver, the SYNC model. SYNC models are maintained in the sender and updated by inferred sync in the receiver. However, you can take local ownership of the model in the receiver. If you do so, they are no longer updated by consolidation, and any updates are discarded.

With custom patterns, the pattern module nodes (those containing the TPL) are not synced, though the pattern nodes are synced to preserve the maintaining pattern relationship.

Collaborative Application Modeling (CAM) produces TPL, which is not synced. However, the BAIs that the TPL builds are synced, as are other inferred nodes, and the pattern nodes are synced to preserve the maintaining pattern relationship. To use CAM on the receiver, export the CAM model from the sender and import it into the receiver. You can do the same with the generated TPL.

Consolidation and clusters

Clusters operate in a similar manner to stand-alone appliances. Stand-alone senders can consolidate to any member of a cluster. When using a cluster as a sender, you configure consolidation on any member UI, but the cluster coordinator sends information to the receiving cluster. The sending cluster can consolidate to any member of the receiving cluster. 

Note

Inferred data consolidation uses port 443 to communicate. The sender must be able to connect to port 443 on the receiver. You must configure any firewalls between senders and receivers to allow this traffic. 

Replacing a scanning appliance (DDD sender) with a BMC Discovery Outpost

The BMC Discovery Outpost can be used to replace a scanning appliance. The only scenario where a BMC Discovery Outpost is not an appropriate replacement is in Restricted (policy) networks, where a line of business requires visibility of only their discovered services and infrastructure. The BMC Discovery Outpost does not provide this restricted view, as it acts as a collector and forwarder of data to the appliance. Advantages of using a BMC Discovery Outpost in Restricted (policy) networks, are in communication, that the BMC Discovery Outpost requires a single HTTPS, web-friendly port (443), and in reduced ownership/management overhead as the BMC Discovery Outpost is self-updating.

Note

When upgrading to BMC Discovery 24.3 (14.1), we strongly recommend that you upgrade by using your existing architecture and testing  version 24.3 (14.1) in a known configuration. 

The following procedure describes the steps required to replace the scanning appliance in a consolidating system with a BMC Discovery Outpost:

  1. Upgrade the scanning appliance to 24.3.
  2. Export the credentials from the upgraded scanning appliance.
  3. Import the credentials into the new BMC Discovery Outpost.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*