Collecting data from external systems

To collect data from external systems

1. Navigate to Administration > Data Collectors > Add Data Collector .

2. In the Name box, provide a unique name to identify this data collector. On the Type list, select Monitor using External Configuration.

3. Provide the following information, as appropriate:

   Field	Description
   Target/Collection Host
   Collection Host (Agent)	Type or select the collection host depending on whether you want to use the Collection Station or the Collection Agent to perform data collection. The collection host is the computer on which the Collection Station or the Collection Agent is located. By default, the Collection Station is already selected. You can either retain the default selection or select the Collection Agent. Note: For this type of data collector, the target host and collection host are expected to have different values.
   Collector Inputs
   Read from Past (#days)	Indicates the number of days for which the past data must be collected and indexed. The maximum amount of past data that can be collected into the system is defined by the maximum data retention period set at Administration > System Settings. By default, this value is set to 0. You cannot search data with a custom time that is set to a duration exceeding the value specified in this field. Recommendation: BMC recommends you to not use a very high value in this field (for example, 365). This is necessary to avoid a very large amount of data collected into the system in a short time.
   Available Ext. Config	Select one of the external configurations available. This list displays the external configurations created under Settings > External Configurations. Based on the kind of data you want to collect, you can select ProactiveNet cell, Infrastructure Management cells, Change Management application, or Incident Management application. Note: If an external configuration contains more than one cell or application selected, this list displays separate instances of each cell or application. A data collector can be used to collect data from only one instance (cell or application). To collect data from more than one instance (cell or application), you need to create corresponding number of data collectors.
   Poll Interval (mins)	Enter a number to specify the poll interval (in minutes) for the log collection. By default, this value is set to 1.
   Start/Stop Collection	(Optional) Select this check box if you want to start the data collection immediately.

    Advanced Options

   Ignore Data Matching Input

   (Optional) If you do not want to index certain lines in your data file, then you can ignore them by providing one of the following inputs: Provide a line that consistently occurs in the event data that you want to ignore. This line will be used as the criterion to ignore data during indexing. Provide a Java regular expression that will be used as the criterion for ignoring data matching the regular expression. Example: While using the following sample data, you can provide the following input to ignore particular lines. To ignore the line containing the string, "WARN", you can specify WARN in this field. To ignore lines containing the words both "WARN" and "INFO", you can specify a regular expression .*(WARN|INFO).* in this field. Sample data Sep 25, 2014 10:26:47 AM net.sf.ehcache.config. ConfigurationFactory parseConfiguration():134 WARN: No configuration found. Configuring ehcache from ehcache-failsafe.xml found in the classpath: Sep 25, 2014 10:26:53 AM com.bmc.ola.metadataserver. MetadataServerHibernateImpl bootstrap():550 INFO: Executing Query to check init property: select * from CONFIGURATIONS where userName = 'admin' and propertyName ='init' Sep 30, 2014 07:03:06 PM org.hibernate.engine.jdbc.spi. SqlExceptionHelper logExceptions():144 ERROR: An SQLException was provoked by the following failure: java.lang.InterruptedException Sep 30, 2014 04:39:27 PM com.bmc.ola.engine.query. ElasticSearchClient indexCleanupOperations():206 INFO: IndexOptimizeTask: index: bw-2014-09-23-18-006 optimized of type: data

   Data Retention Period (in days)

   Indicates the number of days for which indexed data must be retained in the system. By default, this value is set to 7. The default value is based on the maximum data retention period specified at Administration > System Settings. You can change this limit to a maximum of 14 days. To increase the limit beyond 14 days, you need to modify the value of the following property: Property name: max.data.collector.data.retention.limit Property location: %BMC_ITDA_HOME%\custom\conf\server\searchserviceCustomConfig.properties After changing the property value, you need to restart the Search component to apply the change. For more information, see Understanding data retention and deletion.

    Tags

   Inherit Host Level Tags From Target Host

   (Optional) Select this check box to inherit your tag selections associated with the target host that you selected earlier. This option is not applicable if you did not select a target host. Note: After selecting this check box, you can further manually select additional user groups. When you manually select additional user groups, both the inherited permissions as well as the manually assigned permissions are applied. To remove the inherited permissions, clear this check box.

   Select Tag name and corresponding value

   (Optional) Select a tag name and specify the corresponding value by which you want to categorize the data collected. Later while searching data, you can use these tags to narrow down your search results. Example: If your are collecting data from hosts located at Houston, you can select a tag name for "Location" and in the value specify "Houston". While searching the data, you can use the tag, Location="Houston" to filter data and see results associated with the Houston location. To be able to see tag names, you need to first add them by navigating to Administration > System Settings. To specify tag names and corresponding values, in the left box select a tag name and then type the corresponding tag value in the right box. While you type the value, you might see type-ahead suggestions based on values specified in the past. If you want to use one of the suggestions, click the suggestion. Click Add to add the tag name and corresponding value to the list of added tags that follow. Click Remove Tag to remove a tag. The tags saved while creating the data collector are displayed on the Search tab, under the Filters panel, and in the Tags section. Note: At a time, you can specify only one value for a tag name. To specify multiple values for the same tag name, each time you need to select the tag name, specify the corresponding value, and click Add. For more information about tags, see Understanding tags.

    Group Access

   Inherit Host Level Access Groups From Target Host

   (Optional) Select this check box to inherit your group access configurations associated with the target host that you selected earlier. This option is not applicable if you did not select a target host. Note: After selecting this check box, you can further manually select additional user groups. When you manually select additional user groups, both the inherited permissions as well as the manually assigned permissions are applied. To remove the inherited permissions, clear this check box.

   Select All Groups

   (Optional) Select this option if you want to select all user groups. You can also manually select multiple user groups. Notes: You can access data retrieved by this data collector based on the following conditions. If user groups are not selected and data access control is enabled: Only the creator of the data collector can access data retrieved by this data collector. If user groups are not selected and if data access control is not enabled: All users can access data retrieved by this data collector. You can restrict access permissions by selecting the relevant user groups that must be given access permissions. To enable data access control, navigate to Administration > System Settings. For more information, see Managing user groups in IT Data Analytics.

4. Click Create to save your changes.