• Spaces
  • Help
    • ×
     
     
    •  
    BMC TrueSight IT Data Analytics 1.0

    Page tree

    This search command groups events depending on the group options (conditions) specified to create a transaction of events. This command can help you correlate event data based on specific fields and provide a single transaction entry containing related event data from multiple systems, tiers, and so on.

    When you run this command, the following three fields are automatically added in the grouped transaction:

    • duration: Provides the time lapse (in seconds) between the first and last event grouped together in the transaction.
    • numentries: Provides the number of events grouped together in the transaction.

    • group_complete: Indicates whether or not the transaction is completed.

    This topic contains the following information:

    For a list of all search commands, see Search commands.

    About the group_complete field

    This field indicates whether or not the transaction is completed. Completion of the transaction is determined by whether or not the group options specified in the search query are met. All transactions for which the group options are not met are considered to be incomplete transactions; for those records, the group_complete field is marked as false.

    When the value of the group_complete field is false, the following scenarios apply:

    • The incomplete transactions are represented differently on the timeline chart depending on whether the search was executed in ascending order or descending order:
      • (Default) Descending order search (group): Incomplete transactions are plotted as the oldest events on the timeline chart (from left to right).
      • Ascending order search (* | head | group): Incomplete transactions are plotted as the newest events on the timeline chart (from right to left).

      Note

      Drilling down on incomplete events on the timeline chart might not produce accurate records.

    • The value of the duration field is the difference between the start time or end time of the transaction and the time at which the first event occurred.

    Syntax

    group <field>+ [<group-options>]*

    In the preceding syntax, the following definitions apply:

    • + indicates one or more similar expressions separated by a comma (,) or a space.
    • * indicates zero or more similar expressions.
    • [expression] indicates it is optional.
    • <field> refers to field names to use for grouping the events into a single transaction.

    • <group-options> refers to the following group options to specify for creating the grouped transaction. These group options are used as conditions for distributing transaction entries.

      ItemSyntaxDescription
      maxspan
      maxspan=<int>(s|m|h)?

      Maximum time span (in seconds, minutes, hours, or days) for grouping the transaction event entries

      Default: 10 minutes with a maximum of 120 minutes

      maxpause

      maxpause=<int>(s|m|h)?

      Maximum interval between the grouped transaction entries

      Default: No limit, but cannot be greater than value of the maxspan option

      maxevents

      maxevents=<int>

      Maximum number of entries in a transaction.

      Default: 100 with a maximum limit of 10,000 events

      startswith

      startswith=<filter-string>
      		Any string in the raw event data that must mark the start of the grouped transaction

      endswith

      endswith=<filter-string>
      		Any string in the raw event data that must mark the end of the grouped transaction

      where,
      ? indicates zero or more similar expressions

      <int> refers to an integer value

    Short examples

    Example 1: Group events occurring in the span of a maximum of 2 minutes.

    ... | group maxspan=2m

    Example 2: Group events occurring with a time gap of 1 minute but limited to a maximum of two event entries.

    ... | group maxpause=1m maxevents=2

    Example 3: Group data entries with the same HOST field value and with no gaps of longer than 5 seconds between two entries.

    ...| group HOST maxpause=5s

    Example 4: Group data entries with a unique combination of userid and sessionid fields. The first and last entries are within the 25-second time gap. All sessions tied with some user ID are also associated in related transactions and the user ID is not required to be present in such log entries.

    ...| group userid sessionid maxspan=25s 

    Example 5: Group data entries with the HOST field where the first entry contains signon and the last entry contains purchase. The startswith and endswith options mark the transaction boundary for log events.

    ...| group HOST startswith=signon endswith=purchase

    Example 6: Create multiple groups by marking the start and end of the group, depending on whether the initial search query exists in the event data.
    ... | group startswith=GET endswith=PUT

    Long examples

    The following sample data and sample indexed data (displayed on the Search tab) will help you understand the examples of using the group command.

    Sample data

    10.1.1.140 - - [11/Jul/2013:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 404 100
    10.1.1.140 - - [11/Jul/2013:15:02:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 150
    10.1.1.141 - - [11/Jul/2013:15:03:52 -0700] "PUT /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 200
    10.1.1.141 - - [11/Jul/2013:15:04:52 -0700] "POST /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 200 100

    Back to examples ↑

    Sample indexed data

    10.1.1.141 - - [11/Jul/2013:15:04:52 -0700] "POST /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 200 100
    HOST=local.bmc.com |ResponseSize=100|COLLECTOR_NAME=u4 |ClientIp=10.1.1.141 |ResponseCode=200 |RequestType=POST|RequestURL=/themes/ComBeta/images/bullet.png
    10.1.1.141 - - [11/Jul/2013:15:03:52 -0700] "PUT /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 200
    HOST=local.bmc.com |ResponseSize=200|COLLECTOR_NAME=u4 |ClientIp=10.1.1.141 |ResponseCode=201 |RequestType=PUT|RequestURL=/themes/ComBeta/images/bullet.png
    10.1.1.140 - - [11/Jul/2013:15:02:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 150
    HOST=local.bmc.com |ResponseSize=150|COLLECTOR_NAME=u4 |ClientIp=10.1.1.140 |ResponseCode=201 |RequestType=GET|RequestURL=/themes/ComBeta/images/bullet.png
    10.1.1.140 - - [11/Jul/2013:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 404 100
    HOST=local.bmc.com |ResponseSize=100|COLLECTOR_NAME=u4 |ClientIp=10.1.1.140 |ResponseCode=404 |RequestType=GET|RequestURL=/themes/ComBeta/images/bullet.png

    Back to examples ↑

    group based on a field

    In this example, you use the command to create transactions per client for analyzing a pattern of requests from the various clients.

    This command groups all HTTP requests (a maximum of 500 events per transaction) for each client based on its IP address.

    Command

    ... | group ClientIp

    Output

    10.1.1.141 - - [11/Jul/2013:15:03:52 -0700] "PUT /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 200 
    10.1.1.141 - - [11/Jul/2013:15:04:52 -0700] "POST /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 200 100
    COLLECTOR_NAME=u4 |ResponseCode=201,200 |HOST=local.bmc.com |duration=60|numentries=2|group_complete=false|ResponseSize=200,100 |RequestType=PUT,POST |RequestURL=/themes/ComBeta/images/bullet.png
    10.1.1.140 - - [11/Jul/2013:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 404 100
    10.1.1.140 - - [11/Jul/2013:15:02:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 150
    COLLECTOR_NAME=u4 |ResponseCode=404,201 |HOST=local.bmc.com |duration=60|numentries=2|group_complete=false|ResponseSize=100,150 |RequestType=GET |RequestURL=/themes/ComBeta/images/bullet.png

    Back to examples ↑

    group based on multiple fields

    In this example, you use the command to create transactions for each client and each request for analyzing a pattern of requests and request types from the various clients.

    This command groups all HTTP requests (a maximum of 500 events per transaction) from a client with the same request type.

    Command

    ... | group ClientIp RequestType

    Output

    10.1.1.141 - - [11/Jul/2013:15:04:52 -0700] "POST /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 200 100
    COLLECTOR_NAME=u4 |ResponseCode=200 |HOST=local.bmc.com |duration=0|numentries=1|group_complete=false| ClientIp=10.1.1.141 |ResponseSize=100 |RequestType=POST |RequestURL=/themes/ComBeta/images/bullet.png
    10.1.1.141 - - [11/Jul/2013:15:03:52 -0700] "PUT /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 200
    COLLECTOR_NAME=u4 |ResponseCode=201 |HOST=local.bmc.com |duration=0|numentries=1|group_complete=false| ClientIp=10.1.1.141 |ResponseSize=200 |RequestType=PUT |RequestURL=/themes/ComBeta/images/bullet.png
    10.1.1.140 - - [11/Jul/2013:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 404 100
    10.1.1.140 - - [11/Jul/2013:15:02:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 150
    COLLECTOR_NAME=u4 |ResponseCode=404,201 |HOST=local.bmc.com |duration=60|numentries=2|group_complete=false|ClientIp=10.1.1.140 |ResponseSize=100,150 |RequestType=GET |RequestURL=/themes/ComBeta/images/bullet.png

    Back to examples ↑

    maxspan

    In this example, you use the command to group events occurring in a span of two minutes.

    Command

    ... | group maxspan=2m

    Output

    10.1.1.140 - - [11/Jul/2013:15:02:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 150
    10.1.1.141 - - [11/Jul/2013:15:03:52 -0700] "PUT /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 200
    10.1.1.141 - - [11/Jul/2013:15:04:52 -0700] "POST /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 200 100
    COLLECTOR_NAME=u4 |ResponseCode=201,200 |HOST=local.bmc.com |duration=120|numentries=3|group_complete=true|ClientIp=10.1.1.140,10.1.1.141 |ResponseSize=150,200,100 |RequestType=GET,PUT,POST |RequestURL=/themes/ComBeta/images/bullet.png
    10.1.1.140 - - [11/Jul/2013:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 404 100
    COLLECTOR_NAME=u4 |ResponseCode=404 |HOST=local.bmc.com |duration=0|numentries=1|group_complete=false|ClientIp=10.1.1.140 |ResponseSize=100 |RequestType=GET |RequestURL=/themes/ComBeta/images/bullet.png

    Back to examples ↑

    maxpause

    In this example, you use the command to group event entries occurring with a time gap of one minute.

    Command

    ... | group maxpause=1m

    Output

    10.1.1.140 - - [11/Jul/2013:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 404 100
    10.1.1.140 - - [11/Jul/2013:15:02:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 150
    10.1.1.141 - - [11/Jul/2013:15:03:52 -0700] "PUT /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 200
    10.1.1.141 - - [11/Jul/2013:15:04:52 -0700] "POST /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 200 100
    COLLECTOR_NAME=u4 |ResponseCode=404,201,200 |HOST=local.bmc.com |duration=180|numentries=4|group_complete=true|ClientIp=10.1.1.140,10.1.1.141 |ResponseSize=100,150,200 |RequestType=GET,PUT,POST |RequestURL=/themes/ComBeta/images/bullet.png

    Back to examples ↑

    maxpause and maxevents

    In this example, you use the command to group events occurring with a time gap of one minute but limited to the number of event entries specified.

    Command

    ... | group maxpause=1m maxevents=2

    Output

    10.1.1.141 - - [11/Jul/2013:15:03:52 -0700] "PUT /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 200
    10.1.1.141 - - [11/Jul/2013:15:04:52 -0700] "POST /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 200 100
    COLLECTOR_NAME=u4 |ResponseCode=201,200 |HOST=local.bmc.com |duration=60|numentries=2|group_complete=true|ClientIp=10.1.1.141 |ResponseSize=200,100 |RequestType=PUT,POST |RequestURL=/themes/ComBeta/images/bullet.png
    10.1.1.140 - - [11/Jul/2013:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 404 100
    10.1.1.140 - - [11/Jul/2013:15:02:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 150
    COLLECTOR_NAME=u4 |ResponseCode=404,201 |HOST=local.bmc.com |duration=60|numentries=2|group_complete=true|ClientIp=10.1.1.140 |ResponseSize=100,150 |RequestType=GET |RequestURL=/themes/ComBeta/images/bullet.png

    Back to examples ↑

    maxevents and maxspan

    In this example, you use the command to group events occurring within two minutes but limited to a maximum of two event entries.

    Command

    ... | group maxevents=2 maxspan=2m

    Output

    10.1.1.141 - - [11/Jul/2013:15:03:52 -0700] "PUT /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 200
    10.1.1.141 - - [11/Jul/2013:15:04:52 -0700] "POST /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 200 100
    COLLECTOR_NAME=u4 |ResponseCode=201,200 |HOST=local.bmc.com |duration=60|numentries=2|group_complete=true|ClientIp=10.1.1.141 |ResponseSize=200,100 |RequestType=PUT,POST |RequestURL=/themes/ComBeta/images/bullet.png
    10.1.1.140 - - [11/Jul/2013:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 404 100
    10.1.1.140 - - [11/Jul/2013:15:02:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 150
    COLLECTOR_NAME=u4 |ResponseCode=404,201 |HOST=local.bmc.com |duration=60|numentries=2|group_complete=true|ClientIp=10.1.1.140 |ResponseSize=100,150 |RequestType=GET |RequestURL=/themes/ComBeta/images/bullet.png

    Back to examples ↑

    startswith

    In this example, you use the command to create multiple groups of events based on whether the string specified exists in the event data.

    Command

    ... | group startswith=GET

    Output

    10.1.1.140 - - [11/Jul/2013:15:02:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 150
    10.1.1.141 - - [11/Jul/2013:15:03:52 -0700] "PUT /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 200
    10.1.1.141 - - [11/Jul/2013:15:04:52 -0700] "POST /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 200 100
    COLLECTOR_NAME=u4 |ResponseCode=201,200 |HOST=local.bmc.com |duration=120|numentries=3|group_complete=true|ClientIp=10.1.1.140,10.1.1.141 |ResponseSize=150,200,100 |RequestType=GET,PUT,POST |RequestURL=/themes/ComBeta/images/bullet.png
    10.1.1.140 - - [11/Jul/2013:15:01:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 404 100
    COLLECTOR_NAME=u4 |ResponseCode=404 |HOST=local.bmc.com |duration=0|numentries=1|group_complete=true|ClientIp=10.1.1.140 |ResponseSize=100 |RequestType=GET |RequestURL=/themes/ComBeta/images/bullet.png

    Back to examples ↑

    startswith and endswith

    In this example, you use the command to create multiple groups by marking the start and end of the group, depending on whether the string specified exists in the event data.

    Command

    ... | group startswith=GET endswith=PUT

    Output

    10.1.1.140 - - [11/Jul/2013:15:02:52 -0700] "GET /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 150
    10.1.1.141 - - [11/Jul/2013:15:03:52 -0700] "PUT /themes/ComBeta/images/bullet.png 
    HTTP/1.1" 201 200
    COLLECTOR_NAME=u4 |ResponseCode=201 |HOST=local.bmc.com |duration=60|numentries=2|group_complete=true|ClientIp=10.1.1.140,10.1.1.141 |ResponseSize=150,200 |RequestType=GET,PUT |RequestURL=/themes/ComBeta/images/bullet.png

    Back to examples ↑

    Notes

    • If in a search query, both startswith and endswith options are specified, then the following conventions apply:

      • If the startswith criteria is not met, the search query start time is used.

      • If the endswith criteria is not met, the search query end time is used.

    • The group maxspan defaults to 10 minutes. Therefore, you might find that transactions are automatically completed after every 10 minutes (group_complete=true).

    • Using a very high value for maxspan and maxevents on data containing high cardinality fields (fields with a large number of values) can result in slow performance.

    © Copyright 2005-2023 BMC Software, Inc. Use of this site signifies your acceptance of BMC’s Terms of Use . BMC, the BMC logo, and other BMC marks are assets of BMC Software, Inc. These trademarks are registered and may be registered in the U.S. and in other countries.