Ingesting historical Remedy data

Use bulk ingestion scripts to get data for historical Remedy Action Request System incidents and change tickets into TrueSight Intelligence. For information on using the plugin to collect live data from Remedy, see Collecting incident and change data from Remedy. Use the historical data for retrospective analysis and understand key trends and patterns in tickets. Use event clustering to perform advanced analytics.

This topic describes the procedure to get historical Remedy Action Request System data into TrueSight Intelligence. Some of the salient features are:

• Remedy Incidents and Changes are available in the form of events in TrueSight Intelligence. Use the predefined templates to get data for more than 100 incident and change ticket fields that are mapped to TrueSight Intelligence fields.
• A TrueSight Intelligence App is created using the app_id specified as part of the properties in the eventDefinition section of the json template.
• Use the Events feature to filter and view incidents or changes in a particular time window. For more information, see Managing events.
• Use the event clustering feature to cluster the tickets based on predefined parameters. For more information, see Clustering event data for advanced analysis.

Prerequisites

• Java 1.8 or later must be installed

• Maven¹

• Remedy API Java SDK¹

¹ Required only if you want to contribute and build code.

Before you begin

Make sure that the field names do not contain any special characters (only _ is allowed). A system validation will prevent ingestion from progressing if the field names contain special characters.

To ingest historical data

Remedy bulk ingestion folder and files on GitHub

1. Navigate to the dist directory.
   $ cd dist

2. Update the changeTemplate.json and incidentTemplate.json files.
   

   The JSON files contain the following sections.

   Configuration required to run the script
   "config": {    "remedyHostName":"xxxx" ---> The host name of the Remedy Action Request System    "remedyPort":"0" ---> The port number used by the Remedy Action Request System. ---> The port number is required only if a custom value was used for the AR Server TPC Port Number in the AR Server Settings.    "remedyUserName":"xxxx", ---> Remedy Action Request System user name    "remedyPassword":"xxxx", ---> Remedy Action Request System password   "tsiApiToken":"xxxx", ---> TrueSight Intelligence account API Token   "startDateTime":"2016-12-31 00:00:00 UTC", ---> The start date used for the date range query   "endDateTime":"2017-12-31 00:00:00 UTC", ---> The end date used for the date range query }
   Event definition
   "eventDefinition": {       "properties": {          "app_id": "CHANGE_ME" ---> The App ID used to tag the events in TrueSight Intelligence       }    }
   Field definition mapping
   "fieldDefinitionMap":{} The default template includes over 100 field mappings each for incidents and changes, which allows you to collect data without any customization. For more information on the default field mappings, see changeDefaultTemplate.json and incidentDefaultTemplate.json . To override the field mapping: Copy the details from the default template. For example, copy the fieldDefinitionMap section from the incidentDefaultTemplate.json. Paste the details from the system clipboard to a notepad file and add, remove, or update the field mapping as required. Advanced configuration To define additional field mappings, refer to readme.md on GitHub . Copy the fieldDefinitionMap details and paste it in the corresponding JSON file. For example, if you copied the fieldDefinitionMap section from the incidentDefaultTemplate.json, you must paste it in the incidentTemplate.json. Source Destination incidentDefaultTemplate.json incidentTemplate.json changeDefaultTemplate.json changeTemplate.json

3. Run the pre-built jar available in the dist folder jar file.

   • $ java -jar remedy-tsi-bulkingestion-script-0.9.8.jar
     Read the output and provide details.

   • (Optional) Use command line arguments (order is not relevant) to provide configuration details.
     For example, $java -jar remedy-tsi-bulkingestion-script-0.9.8.jar <incident> <change> <exportincident> <exportchange> <silent> <retry> <loglevel>
     Use debug as command line parameter to enable debug mode.
     For example, $java -jar remedy-tsi-bulkingestion-script-0.9.8.jar debug

     Command line argument	Description
     incident	Ingest incident tickets.
     change	Ingest change tickets.
     exportincident	Export the ingested incidents to a CSV file.
     exportchange	Export the ingested change tickets to a CSV file.
     silent	Start the script based on the provided command line arguments in silent mode, and do not prompt for further inputs.
     retry	Create and dump a csv file with invalid events, and read from the csv to retry ingestion of these events.
     loglevel	Set the logging level. For example, info, error, debug.