Consolidating from an unpatched to a patched appliance
The latest patch release (11.0 patch 6) has created potential loss of data between unpatched scanning appliances and patched consolidating appliances.
In general consolidation works across supported versions of Discovery and we do not insist that scanning appliances and consolidating appliances are the same versions. However it is now the case that if your consolidating appliance is on the latest patched version, and a scanning appliance is not yet patched to the latest patch version some data will be lost in consolidation.
The simple solution right now is to upgrade any scanners to the latest patch level; these are:
- 11.0 patch 6
- 11.1 patch 8
- 11.2 patch 6
- 11.3 patch 5
Consolidation refers to the centralization of discovery data from scheduled or snapshot scans on multiple scanning appliances to one or more consolidation appliances. You might want to use consolidation in the following scenarios:
- Firewalled environments—When an environment is divided by firewalls so that a single appliance is unable to reach all parts of the network, a scanner can be situated on each section of the network blocked by a firewall. The scanners can all feed back data to a central consolidator.
- Restricted (policy) networks—Certain lines of business might enforce policies on the control of IT infrastructure in their environments. Where such policies limit or prohibit access, scanners can be deployed which all feed back data to a central consolidator.
- Restricted (time) scanning windows—Where a discovery window is short, a single appliance might be unable to complete a scan of a large range of IP addresses during the permitted time. Sharing the IP addresses between multiple scanners means each smaller scan can be completed in less time, and the results can be consolidated and viewed on the consolidator. You may consider using a cluster in this situation.
In each of these situations, multiple scanners can be deployed, and their data consolidated into a central consolidator. The consolidator is then used for reporting and provides a coherent view of the entire scanned network. A consolidator must be set as one which accepts connections or feeds from scanners. Scanners must in turn register with a consolidator.
Consolidators can also scan
Any consolidation appliance can also be used to perform discovery in its own right.
IP address ranges
Although consolidation can be used to scan a firewalled environment, it is essential that the IP address ranges scanned by each scanner belong to the same IP address space. That is, if two scanners scan the same address, they must both reach the same device. If the IP address spaces are not consistent across all the scanners, information on the consolidator can be missing or incomplete.
This restriction applies only to the addresses scanned by the scanners; if discovery targets possess other IP addresses, there is no need for them to belong to a consistent IP address space.
Consolidator—The main purpose of the consolidator is to report on data consolidated from a number of other scanners. A consolidator can also be used to perform discovery in its own right.
Scanner—The scanner appliance also operates as a normal appliance. The only difference is that it constantly sends discovery data to the consolidator. After setting up, this process is transparent to the user. A scanner must request and be approved on a consolidator appliance before it can send any data to the consolidator. This is described in Approving or rejecting a scanner request. A scanner can send consolidation data to more than one consolidator.
On the consolidator user interface, the Currently Processing Runs tab shows any local scans and any consolidation runs in progress. The Currently Processing Runs is described in The Discovery Status page.
The consolidator's service pack release must be the same or greater than the scanner. This is checked when you test the scanner-consolidator connection and when the scanner periodically checks that the consolidator is still accessible.
- An 11.0 consolidator can accept data from a 9.0, 10.0, 10.1, 10.2 and 11.0 scanner
- An 11.1 consolidator can accept data from a 9.0, 10.0, 10.1, 10.2, 11.0 and 11.1 scanner
- An 11.2 consolidator can accept data from a 9.0, 10.0, 10.1, 10.2, 11.0, 11.1 and 11.2 scanner
- An 11.3 consolidator can accept data from a 9.0, 10.0, 10.1, 10.2, 11.0, 11.1, 11.2 and 11.3 scanner
If you try to consolidate to an earlier version, warning messages are shown in the UI.
What is consolidated?
The consolidated data is the BMC Discovery Directly Discovered Data (DDD) nodes including the data collected by the patterns. The data inferred by the scanners, for example, Software Instance nodes, is not consolidated, but the consolidator will infer it again (based on its pattern configuration).
TKU release, patterns, CSV imports and consolidation
The TKU release package and custom patterns that are loaded on the scanning and consolidators must be the same in order to infer the same data, for example, Software Instance nodes. This is not enforced in any way by the system.
The data imported via CSV in a scanner will not be consolidated. It has to be imported into all other appliances too.
Missing information when patterns run commands on other hosts
When a host is discovered and patterns are triggered which run commands on a second host, the DDD on both hosts is updated. When the original host is consolidated, the DDD on the second host is not available to the patterns that trigger on the consolidator. When the second host is consolidated, the DDD created on it when discovering the first host is not included. Consequently the consolidator will always report that the information from the second host is unavailable. The error "Request for information not part of the consolidated data" will be reported in the consolidated DiscoveryAccess. This can lead to missing nodes (licensing Detail, SoftwareComponents, and so on) and relationships on the consolidator. To work around this behavior, scan the original host from the consolidator.
Configuring consolidation is a two step procedure. Initially the appliance which is to be the consolidator must be set as a consolidator, and then one or more scanners register with the appliance. To configure consolidation you need the permissions detailed in Consolidation Permissions.
Consolidation and clusters
When using standalone scanners to consolidate to a cluster, use the IP address of the coordinator (displayed in the cluster UI) as the consolidation target. If you subsequently change the coordinator of the target cluster, you must update the consolidation configuration on the scanner.
When using a cluster as a scanner, you can configure consolidation using any member UI, but only the coordinator of the cluster sends information to the consolidator. The coordinator of the target cluster must be specified as the consolidation target.
Firewalls and consolidation
Consolidation uses port 25032 to communicate. The scanner must be able to connect to port 25032 on the consolidator. You must configure any firewalls between scanners and consolidators to allow this traffic. For clusters that act as scanners you must open port 25032 on all members. For clusters that act as consolidators you must open port 25032 on the coordinator, but if you change the coordinator you must open port 25032 on the new coordinator.
To set an appliance as a consolidator
- From the main menu, click the Administration Settings icon, and then select Discovery Consolidation.
You cannot use consolidation if the appliance is named Discovery_Appliance. A warning is displayed, including a link to where you can change the appliance name.
- On the Consolidation page, click Set as Consolidation Appliance.
The appliance is now configured as a consolidator.
To set an appliance as a scanner
- From the main menu, click the Administration Settings icon, and then select Discovery Consolidation.
On the Consolidation page, click Set as Scanning Appliance.
This dialog enables you to specify a consolidation target. Enter or edit the following information in the dialog:
The name of the scanner. Names must be unique in the consolidation network and you cannot consolidate a scanner with the default name, Discovery_Appliance. The name is taken from the Administration > Appliance Configuration > Identification page. For more information, see Initial configuration. A change link is provided that displays the Identification page. On the identification page, you can change the name of the appliance. You can consolidate only those appliances that have unique names.
The address of the consolidator. This can be specified as one of the following:
- Hostname or FQDN
IPv4 or IPv6 address
You can supply credentials for the consolidation appliance in this dialog. If you supply valid credentials here, the scanner is approved automatically.
The user name for a user on the consolidator. This user must have appropriate permissions to approve the connection of the scanner to the consolidator.
The password for the user on the consolidator.
If the target consolidator is an earlier version than the scanner, you are warned that the consolidator version is too old.
If you supplied valid credentials for automatic approval on the consolidator, the scanner is now configured.
If you didn't supply credentials, the consolidator must approve the request. This is shown on the scanner:
To add an additional consolidator
A scanner can send consolidation data to more than one consolidator.
- Click the Add new Consolidation Appliance button.
The Add New Consolidation Appliance dialog is displayed. This is described above.
- Enter the details of the consolidator and, if required, the username and password for automatic approval.
- Click Submit to apply the changes.
Approving or rejecting a scanner request
After a request (without automatic approval) has been made from a scanner, it requires approval on the consolidator.
- From the main menu, click the Administration Settings icon on the consolidator, and then select Discovery Consolidation from the Discovery section.
In the following example, the "de-32.tideway.com" appliance has requested to become a scanner.
- Do one of the following:
- To accept the appliance connection, click Approve.
- To reject the request, click Reject.
When you do this, the connection is deleted from the consolidator, and when no connections remain, the scanner reverts to a nonconsolidated machine.
When consolidation is running
After consolidation has been set up, whatever scanning takes place on the scanner is automatically sent to the consolidator as soon as possible after the scan of an endpoint is complete. On the consolidator, runs are displayed that are marked specifically as consolidation runs and can be viewed from the Discovery Status page.
Discovery must be running on the consolidator for consolidation to take place. If Discovery is not running, the consolidator will refuse to accept data from the scanner. The scanner will attempt to resend data later. Also, if Discovery is stopped on the consolidator, it will stop consolidating any data it has already received.
Canceling consolidating discovery runs
You can cancel a consolidating discovery run from the scanner or from the consolidator. Where possible, always cancel the discovery run on the scanner, by selecting the discovery run on the Discovery Status page of the scanner and clicking Cancel Runs.
Canceling the discovery run at the scanner enables the consolidator to finish receiving data from the scanner. This stops the scan rather than the consolidation so that the two appliances' data remains consistent.
Canceling a consolidation run on the consolidator stops the consolidation, although the scan continues on the scanner. This leads to inconsistencies between the data on the two appliances. Where possible, always stop the scan on the scanner and allow the consolidation to run to completion.
If you must cancel a consolidation run from the consolidator, you can do so by selecting the discovery run on the Discovery Status page of the consolidator and clicking Cancel Runs. If there are problems canceling the consolidation run, a status message is displayed.