tw_imp_csv
The tw_imp_csv
utility enables you to search the datastore for nodes of a specified kind that have keys matching rows in the supplied csv data. Where the keys match, the node is updated, or deleted and recreated (depending on the options selected).
Recommendation
Use the BMC Discovery user interface to perform the functionality provided by the tw_imp_csv
command line utility (see Importing CSV data). If you choose to run the utility, read the documentation in this section to learn its usage and to understand the risks and potential impact on your environment.
Incorrect usage may result in data loss
Before using the tw_imp_csv tool you should fully understand the system taxonomy and the changes that you are going to make to your data. Using the tw_imp_csv tool incorrectly can cause irreparable damage to your data. The data you submit using this tool is applied directly to the production data without any validation.
Always back up your datastore before using this tool.
Do not import the following node kinds
- You must never import DDD nodes.
- You should avoid importing Host nodes and other system maintained nodes. If in doubt, contact Customer Support.
To use the utility, type the following command at the $TIDEWAY/bin/
directory:
tw_imp_csv [options] files
where:
- files is a list of csv files to import.
- options are any of the options described in the following table and the common command line options described in Using command line utilities.
Command Line Option | Description |
---|---|
| Specifies that only new nodes will be created; existing nodes are not updated |
| (Do Not Use) Specifies the name of the datastore service. The default is tideway. |
| Specifies that matching nodes will be deleted; no other nodes will be affected |
| Specifies a field delimiter character |
| Specifies an escape character |
| Disables the validation checks that are performed against the taxonomy to check the specified node-kind and attribute/relationship keys. When you use this option, all keys, attributes and relationship links will be left as strings. |
| Displays a comma-separated list of key columns to use for the data key |
| Displays the kind of node to create, update, or clear |
| Specifies a line termination string |
| Specifies the name of the partition to query, where name can be set to All for all partitions (the default). |
| Specifies a quote character |
| (Do Not Use)Specifies the name of the search service |
| (Do Not Use)Specifies the name of the taxonomy service |
| Specifies that only existing nodes will be updated; new nodes will not be created |
| Specifies to hide the name of the uploaded file |
| Specifies the BMC Discovery user name to use to import data. If no name is specified, you are prompted for one. |
| Specifies to display informational messages while processing. This is useful for diagnosing errors |
User examples
In the following examples, enter the commands on a single line. Line breaks are provided to make the examples easier to read.
Freeing rack space for applications
To free rack space for other applications, some hosts have been moved from the 'Campus' data centre to the newly acquired 'Firehouse' data centre. Discovery and Reasoning have handled the IP address and subnet changes but the Host nodes are still linked to the wrong location. Here is the CSV file to process, called firehouse_move.csv
:
name,#HostInLocation:HostLocation:LocationOfHost:Location.name egon,Firehouse ray,Firehouse peter,Firehouse
BMC Discovery processes the CSV file with the following utility at the command line:
$ tw_imp_csv --username=system --password=system --kind=Host firehouse_move.csv
The utility reads the file called firehouse_move.csv
line by line. It uses the first line to name the columns. The first column is called 'name', which doesn't begin with a '#' character so it is treated as an attribute name. The second column does begin with a '#' character so it is treated as a specification for some relationships.
No explicit key has been specified so the first (and in this case, only) attribute name is taken to be the key.
Next, the utility reads the second line. The first field, egon
, is in the name
column which was selected as the key earlier. So the script uses the search service to find a node of kind Host
(from the --kind
command line option) that has a name
attribute equal to egon
. It finds exactly one node matching that search. If it had not found that node, it would have been created. If it had found multiple nodes, an error would have been reported and processing would continue with the next line, NO nodes would have been updated.
Having found the node, it updates it using the other fields on the row. Were there any other attribute columns in the the file, the utility would have used these to update the node before looking at the relationships.
The file has only one relationship column. The name is #HostInLocation:HostLocation:LocationOfHost:Location.name
.
The utility searches for a Location node with a name
attribute equal Firehouse
, this row's value for the column. Having found the Location node, the utility creates a HostLocation
relationship to it with the Host node playing the HostInLocation
role and the Location playing the LocationOfHost
role.
Then the utility processes the second and third data rows, updating the ray
and peter
nodes with the new location.
Installing a remote Windows proxy
A new host has been installed in the Firehouse data centre. Due to pressure from the organisation's E-services Protection Adviser, there is now a firewall preventing discovery of hosts on that site. Until a remote Windows proxy can be installed, the Firehouse system administrators have been sending us spreadsheets with the changes.
The CSV file new_host.csv
looks like the following:
name,fqdn,#HostInLocation:HostLocation:LocationOfHost:Location.name,#ITOwnedItem:ITOwner:ITOwner:Person.name,ip_addrs,#HostOnSubnet:HostSubnet:Subnet.name winston,winston.example.com,Firehouse,"Melnitz,J","['10.3.4.1','192.168.101.45']","['10.0.0.0/8','192.168.101.0/24']"
BMC Discovery processes the CSV file with the following utility at the command line:
$ tw_imp_csv --username=system --password=system --kind=Host --create new_host.csv
The syntax is nearly the same as in the previous example. The differences are that the filename has changed because we are processing a different file and we are using Create Only mode.
As in the previous example, the utility searches for Host nodes with a name
attribute equal to winston
. This time, because it is in create mode, the utility checks that there are no matching nodes. If there were, the utility would report an error and skip the row.
Next, the utility creates a new node. It populates the attributes of the node from the non-relationship fields in the data. The ip_addrs
field is a list and the value starts with a '[' character so it is converted into a list. The new node has attributes:
name = 'winston'
fqdn = 'winston.example.com'
ip_addrs = [ '10.3.4.1', '192.168.101.45' ]
Then the utility adds the relationships. The location relationship column is processed in the same way as in Example 1. The column called #ITOwnedItem:ITOwner:ITOwner:Person.name
creates an ITOwnedItem:ITOwner:ITOwner
relationship to a Person node with a name equal to Melnitz,J
. The quotes around that field are needed because the field contains a comma.
After the quote, the #HostOnSubnet:HostSubnet:Subnet.name
field begins with a '[' character. This field is converted into a list. Then, for each item in the list, a HostOnSubnet:HostSubnet:Subnet
relationship is created to a Subnet node with a name
attribute equal to that item. So the new host has one relationship to the 10.0.0.0/8
subnet and one to the 192.168.101.0/24
subnet.
Comments
Log in or register to comment.