Page tree

Skip to end of metadata
Go to start of metadata

This topic contains the following sections:

Overview

Connectors load entities, relationships between entities, and time-varying data about these entities into BMC TrueSight Capacity Optimization. We explain how BMC TrueSight Capacity Optimization connectors identify the entities that they import from the data source. Below, we will talk specifically about System entities, but the same logic applies to all kinds of entities.

Identifying systems using DS_SYSNM

Recognizing systems during different runs

A connector needs to follow some convention to identify systems, so that each time it runs, it can tell which systems in its data source were seen before and which ones are new.

If the data source contains any obvious unique identifying information, then the connector need only create a unique string as the DS_SYSNM property for the system. The data source might use different pieces of information, all of which need to match exactly (for example, an asset identifier and a database name); the connector developer is responsible for constructing a single DS_SYSNM string from these pieces in a unique and systematic manner.

When loading data, the BMC TrueSight Capacity Optimization connector infrastructure will automatically check to see if another system with the same DS_SYSNM is already loaded by this connector instance. If so, BMC TrueSight Capacity Optimization will associate the new data with the existing system. If not, then BMC TrueSight Capacity Optimization will create a new system. All the systems loaded by a connector instance are tracked in a lookup table. You can view the lookup table for any connector instance in the Admin section of the console. There are separate lookup tables for entities of type System, Business Driver, and Domain. Each table lists the unique identifier string for each entity of that type (for example, DS_SYSNM for Systems) loaded by the connector instance.

Note

This DS_SYSNM field is different from the SYSNM field that is shown as the "Name" of the System in the BMC TrueSight Capacity Optimization console. The latter is a convenient, recognizable name for interacting with BMC TrueSight Capacity Optimization and need not be unique. For example, a server might have a simple, short name, while a connector might identify it uniquely by using its fully qualified domain name (FQDN) as DS_SYSNM.

You can control whether BMC TrueSight Capacity Optimization should use only the name of the host as the DS_SYSNM string for comparisons, or whether it should also consider the Internet domain name (FQDN), using an option in the "Loader configuration" section of the configuration. (See Handling ETL lookup name). By default, the Internet domain name (for example, .bmc.com) is automatically included.

Sharing lookup between different connectors

Often, you need to import information about the same entity from two different data sources. For example, a monitoring system might have performance data about a set of computers, while an asset management database might have administrative information about these same computers.

In BMC TrueSight Capacity Optimization, this situation implies that two different connectors need to load information about the same entities. Because two different connector instances may run on different schedules, this also implies that a new entity in either data source might be discovered first by its connector instance, so either connector instance might end up first loading the entity.

You can configure two or more different connector instances to load the same set of entities. You should configure them to share lookup. When a set of connector instances are configured to share lookup tables, BMC TrueSight Capacity Optimization will consider all the systems loaded by any of them when deciding whether a system is new or an already existing one.

In order to share lookup between two different connectors, the two connectors need to use compatible conventions for attributes to use for identifying entities. BCO provides connector developers with several mechanisms to specify attributes to identify entities loaded earlier, and also to match entities loaded by other connectors. If the connectors are not completely compatible, then sometimes duplicate entities can be loaded in BCO. If this happens, then manual reconciliation may be needed in order to discard one of them.

Simple Lookup and Multiple Lookup

If different connectors are using the DS_SYSNM described above to identify entities, and if they are all constructing the DS_SYSNM string using the exact same method, then sharing the lookup table will work, because all of the lookup strings in the shared lookup table will identify the loaded entities correctly.

But sometimes the different data sources for the different connectors do not have the same unique naming convention to identify the entities. For example, an asset database might use an asset ID to identify servers, while a performance monitoring system might know them by their FQDN instead.

In this case, it is not possible for their connector instances to use the same DS_SYSNM. For this situation, the BMC TrueSight Capacity Optimization connector infrastructure offers connector developers a facility called multiple lookup, where a sequence of field combinations is specified by the connector to match entities.

Advanced connectors can be written in a way that allows for multiple ways to identify entities, so that lookup can be shared even with other connectors that don't have access to all the identifying information for their entities.

From the BMC TrueSight Capacity Optimization administrator's perspective, you don't have to know how multiple lookup works: you simply configure them to share lookup, and as long as the various connectors sharing the lookup table are written in a compatible way, they will work correctly.

From the connector developer's perspective, you need to know more details about how to use this multiple lookup feature.

Strong and weak lookup entries

If different data sources might have different sets of identifying information, then it makes sense to allow for this possibility by defining several different methods of identifying entities. Each method is called a "lookup entry", which is a set of named fields.

In Simple lookup, as described above, the connector specifies only a single lookup field named DS_SYSNM. But for Multiple lookup, the connector can specify any number of lookup fields with their own names. A set of lookup fields forms one lookup entry. An entry matches an entity only if all of the named fields in it match the corresponding values loaded into BMC TrueSight Capacity Optimization earlier. This scheme is backward compatible with existing connectors that use Simple lookup. For "Simple Lookup", the connector specifies DS_SYSNM as the property name, and the loader translates this to a lookup entry with a single field named DEFAULT.

There are two kinds of lookup entries that can be defined by the connector:

  • A strong lookup entry is one that is used by this connector to identify entities it loaded earlier. These entities are looked up in its own lookup table, in a way similar to DS_SYSNM. There can be one or more strong lookup entries defined by a connector. If there are more than one, then they are defined in a sequence. The first entry in the sequence is tried first.
  • A weak lookup entry is used by a connector, not to identify entities it loaded earlier, but to match entities loaded by other connectors. Multiple weak lookup entries are also defined in a sequence, and they are tried in that sequence.

The loader's entity lookup process tries all the strong lookup entries first, then all the weak lookup entries. This means that if there are no other connectors loading the same entities, only the strong lookup entries are tried.

Whenever a match is found, BMC TrueSight Capacity Optimization maintains all the lookup entry values that are loaded by all the connectors sharing lookup. Entries will tend to match other connectors, provided the other connectors have access to the same fields, and provided they supply these fields in weak lookup entries.

"Weak" lookup entries are never tested on the connector's own lookup table; they are used only to search the shared lookup table when no strong lookup combination has been found.

When a weak lookup entry is found in the shared lookup table, the loader has found a "tentative" match for an existing BCO entity. The loader will then verify all of the strong lookup field combinations populated by the connector again on the tentative entity, to ensure that different strong entries don't match the same BCO entity. The match is positive only when all the strong entries uniquely identify the entity.

This multiple lookup facility makes it likely that different connectors can correctly identify the same entities from different data sources. There is still a possibility of missing matches and creating two entities by mistake. These will still have to be manually reconciled in the usual way.