Configuring Splunk Web Logs Extractor
- NCSA-compliant: Splunk is capable of automatically detect NCSA-compliant web log formats (generated by server such as Apache) and it assigns to the indexed data one of the source types
- access_combined
- access_combined_wcookie
- access_common
- Microsoft Internet Information Services: Splunk is capable of indexing IIS generated logs and there are some source types that can be used to make Splunk recognize fields in the IIS logs
- iis
- iis-X (where X is any version of IIS, e.g. iis-7)
The connector leverages the above-mentioned source types definition, so it is crucial that they are not modified in the Splunk instance the connector will interact with. The connector only imports data labelled with the supported source types.
Additionally the connector provides the possibility to aggregate web volumes at the cluster level, specifying some aggregation rules on the basis of single host names.
Full list of configuration properties
The following are the specific settings valid for connector "Moviri – Splunk Unix-Windows Extractor", they are presented in the "Splunk – Unix and Windows" configuration tab.
Property Name | Value Type | Required? | Default | Description |
Whitelist for indexes | String | No |
| A semicolon-separated list of Splunk indexes[1] that represents the only indexes where to extract data from. * wildchar is supported on index names. Empty means no filtering. |
Blacklist for indexes | String | No |
| A semicolon separated list of Splunk indexes that represents the indexes to be excluded from the data extraction. |
Whitelist for hosts | String | No |
| A semicolon separated list of hosts (i.e. web servers) that represents the only hosts whose data need to be extracted. Each item of the list can be a regular expression. |
Blacklist for hosts | String | No |
| A semicolon separated list of hosts (i.e. web servers) that represents the hosts whose data need to be excluded from the extraction. Each item of the list can be a regular expression. |
Import data for | String | Yes |
| Specify which data the connector has to load into CO:
|
Clusters definition rules | String | Yes |
| Semicolon separated list of rules to be applied to aggregate hosts level data into cluster level data |
-- Following properties are repeated for each rule specified in “Clusters definition rules” – | ||||
Regex for [ruleX] | String | Yes |
| Regular expression to be applied on hosts. If it matches the host is included in the cluster whose name is specified in the next property. |
Cluster name for [ruleX] | String | Yes |
| The cluster name. It can be dynamic if regex capturing group are used. See example below. Capturing group are referenced with the syntax %GROUPN (N is the capturing group index). |
In order to facilitate the application of cluster rules, two examples are provided. Consider to have the following web servers: ServA001, ServA002, ServB008, ServB009, ServC001.
EXAMPLE: Clusters definition rules: RuleA;RuleB;RuleC
- RuleA
- Regex: ServA.+
- Cluster name: ServiceA
- RuleB
- Regex: ServB.+
- Cluster name: ServiceB
- RuleC
- Regex: ServC.+
- Cluster name: ServiceC
Resulting clusters and hosts
- ServiceA
- ServA001
- ServA002
- ServiceB
- ServB008
- ServB009
- ServiceC
- ServC001
Datasets managed by this integration
An ETL task that uses the ‘Moviri – Splunk Web Logs Extractor’, will allow you to import:
- the WKLDAT (Business Drivers data) dataset
- the WKLWEB (Business Driver Web) dataset
- the OBJREL (Object Relationships) dataset
You should not be changing the associated datasets, unless for example to avoid importing Object Relationships data.
BMC TrueSight Capacity Optimization entities and metrics
The connector will create a Business Drivers of type “Web business driver” for each web server or cluster imported. The following is the list of metrics populated both for web servers and clusters.
Metrics with * were previously custom, remapped as standard after v 2.3.00
Splunk Metric | BMC TrueSight Capacity Optimization Metric | BMC TrueSight Capacity Optimization Metric Description |
Count of hits on the web server/cluster | WEB_TOTAL_HITS | Total Hits |
Count of hits that generated an http status between 400 and 499 (e.g. 404 not found) | WEB_TOTAL_HITS_BADREQ | Total Bad Request Hits |
Count of hits that generated an http status between 500 and 599 (e.g. 505 server error) | WEB_TOTAL_HITS_ERROR | Total Error Hits |
Count of hits that did not generate an http status between 400 and 599 (e.g. 202 accepted) | WEB_TOTAL_HITS_OK | Total Hits OK |
Amount of downloaded bytes by clients, in terms of average byte rate. | WEB_TRANSF_BYTE_RATE | Bytes Rate Transferred |
A label for the web server/cluster | INFO | General Info |
The number of web servers included in the cluster (applicable to clusters only) | WEBSERVER_NUM_C | Number of Webservers included in layer |
Comments
Log in or register to comment.