Using TPL to enrich discovered data

While standard discovery processes can provide most key information, there are times when you might want to track information that is not available to BMC Discovery. A good example of this type of information is the exact location of a Host. In most situations, there is nothing discoverable on a host that allows you to know that it is located in U23 of Rack 15 in Data Center 2 of the Houston campus. While the most common example of this is to track the location of hosts, similar approaches could be employed for relating hosts to organizational units, or owners.

BMC Discovery makes it very easy for you to extract information from external data sources and include it in its own data. Commonly, customers can use these capabilities to:

Relate hosts to locations using a host naming scheme
Read location information from a file or registry setting
Use a table in a pattern to relate subnets to locations
Query an SQL database for location information

These are described in the sections below.

In the TPL code (patterns) snippets, logging and basic error checking is shown in line with BMC Software recommendations. When you are developing patterns, you may wish to run them outside a normal discovery run against previously scanned host. For information on doing this, see Executing patterns manually.

Location using a naming scheme

The template_host_location template pattern shows you how to relate hosts to locations based on hostname. This pattern relies on a naming scheme in which hostnames contain a location identifier such as lon for London, or hou for Houston.

For a description of all template patterns supplied on the appliance, see Pattern templates.

Reading location from a file

A common way of incorporating location information into a host is to store a file in the host's file system. The template_si_version_xml_file template pattern demonstrates taking information from an XML file to version an SI. The same approach can be used to extract location information from an XML file.

If the information is stored in an XML file, use xpath as shown in the template pattern. The TPL xpath functions are described in the TPL Guide. In the case where information is placed in a flat file, a simple regex is the most effective method of extracting the location. The TPL regex functions are described in the TPL Guide.

The full specification of the regular expression syntax used in BMC Discovery can be found on the Python website. A simpler introduction is also available.

Here is an example XML snippet for a file:

<?xml version="1.0"?>

<config>
   <physical>
      <location="Houston">
      ...

Here is the TPL which retrieves the file and uses xpath to obtain the location name from the file:

    // Get config file and check that it was retrieved successfully
    cfg_file := discovery.fileGet(host, cfg_file_path);
    if cfg_file then

        // Extract location from conf file
        location_name := xpath.evaluate(cfg_file.content, raw '/config/physical/@location')[0];
    end if;

    if not location_name then
        stop;
    end if;

    // Find an existing Location node for the name. The search actually returns a list.
    location := search(Location where name = %location_name%);

    if not location then
        log.error("No Location node for location name %location_name%.");
        stop;
    end if;

Now you can create or update the relationship between the Host and Location node. The following code snippet uses the model.uniquerel function to create or update the relationship.

    // Relate Host to Location. Using "uniquerel" rather than "rel" means that
    // any existing Location relationships between this Host (the first 
    // parameter) and any Location nodes other than the one given (the second
    // parameter) are removed. If a host has changed location, this keeps the
    // model up-to-date.
    //
    // location is actually a list returned by the the search, but the relationship
    // creation functions are happy to take lists as their parameters.

    model.uniquerel.Location(
       ElementInLocation := host, 
       Location          := location
    );

See the template_si_version_xml_file template pattern for an example of how these statements are used in the pattern.

Reading location from a registry key

Some administrators might prefer to place information in a registry setting rather then the files used in the previous section. The template pattern template_si_version_registry demonstrates extracting information from the registry to version an SI. The same approach can be used to extract location information from the registry.

    // Get the registry key
    location_name := discovery.registryKey(
                        process, raw "HKEY_LOCAL_MACHINE\HARDWARE\DESCRIPTION\System\Location");

Using a table in a pattern to relate subnets to locations

Subnets can be used to identify locations of hosts. You can extend the template_host_location pattern to map subnets to locations. The following TPL snippet shows how you could use a table to map the subnet to hosts. You could also hard code a mapping of hostnames to locations if that information is available. See TPL Guide for more information on tables.

table SubnetLocations 1.0

    "10.10.10.0/24"  -> "London Victoria";
    "10.10.11.0/24"  -> "London Egham";
    "172.17.94.0/24" -> "Houston";
    "172.18.95.0/24" -> "Dallas";
    default -> "unknown";

end table;

The following TPL snippet shows the search from the host to associated subnet or subnets.

    subnet_list := search(in host traverse DeviceWithAddress:DeviceAddress:IPv4Address:IPAddress
                                  traverse DeviceOnSubnet:DeviceSubnet:Subnet:Subnet
                                    where ip_address_range defined
                                    show ip_address_range);

    list_size   := size(subnet_list);
    log.debug("subnet_list size is %list_size%");
    
    // The search returns a list. We pick the first IP range for which there
    // is an entry in the SubnetLocations table

    location_name := none;

    for ip_address_range in subnet_list do
        if ip_address_range in SubnetLocations then

            // Matched, so set the location_name and break out of the loop
            location_name := SubnetLocations[ip_address_range];
            break;
        end if;
    end for;

Querying an SQL database for location information

The template_sql_asset_integration template pattern is supplied with BMC Discovery which enables you to extract location information from an SQL database. To query a database you must define the asset database in the pattern and activate the pattern. After activation, this creates an integration point into which you add connection information. For information on using integration points to connect to databases, see Discovering database content.

Automatically creating Location nodes

The pattern examples above have shown a number of ways of determining locations and how to create the relationship between host and existing Location nodes. The following code snippet shows how you can create or update a location node, asserting that the location exists.

    // Create or update a Location node for this. For a list of attributes for
    // Location nodes, see Administration > Taxonomy. Every attribute used on 
    // the node in the "location_row" variable must be returned by the query's
    // SELECT statement.

    log.debug('Setting location of %host.hostname% as %location_row.address%');
    
    location := model.Location(
      key     := text.hash("%location_row.name%/%location_row.address%"),
      name    := location_row.name,
      address := location_row.address
    );