Receiving data over an HTTP or HTTPS connection
To receive data over an HTTP/HTTPS connection, you need to create the Receive over HTTP/HTTPS data collector. This data collector acts like an endpoint that allows you to receive application data (or events) into TrueSight IT Data Analytics by using the HTTP or HTTPS protocol.
When you use the HTTPS protocol, the data collector uses SSL based authentication. You can optionally provide an access token that is shared between the HTTP client and TrueSight IT Data Analytics. This token is used for authenticating the client sending data into TrueSight IT Data Analytics.
Note
While sending requests to TrueSight IT Data Analytics, ensure that you do not break a single event into multiple requests.
Creating this data collector requires minimal inputs. Data sent from the client to TrueSight IT Data Analytics is encrypted.
The following information can help you understand the steps for creating this data collector and other aspects related to the creation process:
All references to the Standalone Agent or Standalone Collection Agent in this document is applicable only if you are using IT Data Analytics version 11.3.01. The latest version released for a Standalone Agent is 11.3.01. Starting from version 11.3.02, no more versions will be released for the Standalone Agent. However, you can make a note of the following information:
- You can continue to use Standalone Agent version 11.3.01 with IT Data Analytics version 11.3.02.
- If you have created Data Collectors using a Standalone Agent in version 11.3.01, the data collection will continue to work with IT Data Analytics version 11.3.02.
- You can also edit the Data Collectors to use PATROL Agent instead of a Standalone Agent in IT Data Analytics version 11.3.02.
Before you begin
If you want to receive data over an HTTPS protocol, ensure that the keystore certificate is configured on the Collection Station (or Collection Agent) that you want to use for the data collection.
To receive data over an HTTP or HTTPS connection
- Navigate to Administration > Data Collectors > Add Data Collector.
- In the Name box, provide a unique name to identify this data collector.
From the Type list, select Receive over HTTP/HTTPS.
Provide the following information, as appropriate:Field Description Target/Collection Host Collection Host (Agent) Type or select the collection host depending on whether you want to use the Collection Station or the Collection Agent to perform data collection.
The collection host is the computer on which the Collection Station or the Collection Agent is located.
By default, the Collection Station is already selected. You can either retain the default selection or select the Collection Agent.
Note: Ensure that the time zone of the collection host is the same as that of the host from which the data comes.
Collector Inputs (Note that data becomes available for searching only after a client sends data to the specified host (on the corresponding port)). Protocol Select HTTP or HTTPS as appropriate.
An HTTPS connection ensures that the data sent from the REST client to TrueSight IT Data Analytics is encrypted.
By default, HTTP is selected.
If you select HTTPS, then you need to provide the following additional inputs: Keystore Type Specify the type of keystore.
For example, JKS, PKCS12, and JCEKS.
By default, this value is set to JKS.
Keystore Path Specify the keystore location.
For example, C:\op\server.store.
Keystore Password Specify password of the keystore file where the certificates are stored.
Key Manager Algorithm Specify the algorithm used by the keystore's key manager factory.
By default, this value is set to sunx509.
Bind address Provide the host name or IP address to which you want to bind .
By default, this value is set to 0.0.0.0Port Provide the port number on which the data collector must listen for incoming connections.
By default, this value is set to 8888.
Access Token (Optional) Specify an access token for authenticating the client sending data into TrueSight IT Data Analytics. The access token can be any string without spaces. This token is shared between TrueSight IT Data Analytics and the REST client.
While sending the message request to TrueSight IT Data Analytics, you need to add an authorization header with the value same as the access token. Name the header as "Authorization" and specify the value as the access token, for example,
Authorization: thesecrettoken
.Data Pattern Pattern Assign a matching data pattern (and optionally date format) for indexing the data.
If the data that you want to collect is in the JSON format, then you need to manually select one of the JSON-related data patterns. Unless you use the correct data pattern for collecting JSON data, key-value pairs in the data are not extracted as fields.
By default the data pattern is set to Free Text without Timestamp. Unlike other data collectors, this type of data collector does not support automatic detection of relevant data patterns. And because this data collector receives events asynchronously, you cannot see a preview of the results right after selecting the data pattern.
After creating the data collector, if you are not satisfied with the search results, then you can assign a new data pattern by manually creating it or by customizing an existing data pattern that closely matches the data that you are collecting (by using the clone feature). You can obtain the sample data for creating (or cloning) the data pattern from the search results on the Search page (data that just got indexed).
The data pattern and date format together decide the way in which your data will be indexed. When you select a data pattern, the matching date format is automatically selected. However, you can override the date format by manually selecting another date format or by selecting the option to create a new date format. By doing this, the date format is used to index the date and time string, while rest of the data is indexed as per the data pattern selected. If you select only a date format, then the date format is used for indexing the timestamp, while the rest of the data is displayed in a raw format in your search results.
For more information, see Assigning the data pattern and date format to a data collector.
Date Format Date Locale (Optional) You can use this setting to enable reading the date and time string based on the language selected. Note that this setting only applies to those portions of the date and time string that consist letters (digits are not considered).
By default, this value is set to English.
You can manually select a language to override the default locale. For a list of languages supported, see Language information for IT Data Analytics
Event Delimiter (Optional) This setting is applicable only when you select Free Text without Timestamp as the data pattern.
All records processed using the Free Text without Timestamp data pattern are assumed to be a single line of data with a line terminator at the end of the event.
Records are distinguished on the basis of the new line separator.
If you want to distinguish records in a custom way, then you can specify a custom string or regular expression in the Event Delimiter box that decides where the new line starts in the data. This string or regular expression must correspond to some text in your data which appears at the beginning of a line.
Start/Stop Collection (Optional) Select this check box if you want to start the data collection immediately. - Click Create to save your changes.
Character set encoding used for sending data
By default, the data collector assumes that the data sent is UTF-8 encoded. To pass another type of character set encoding, you need to include the following line in the message header of the client. In the following line, replace utf-8
with the character set encoding that you want to pass.
Content-type: application/plain; charset=utf-8
Limit for collecting JSON events
You can only collect JSON events (data between the opening and closing braces) upto 10 KB in size. Any JSON event that exceeds this limit is automatically rejected. Therefore, events that exceed the default limit are not indexed and are not searchable. For more information, contact BMC Support.
Comments
Log in or register to comment.