Running a search query

From the Run Forensic Search Query page, you can select the parameters of a search, and launch the search background process. The following image displays the Run Forensic Search Query page when a background query is not running:

If a background query is in process, the top-level Query page is displayed. You must wait or terminate the background query to view this page.

To run a search query

1. Navigate to the Reports > Query page and click the Generate button to access the Run Forensics Search Query page.

   Success

   Tip

   You can also access this page in one of the following ways:

   • Click the Run Query hyperlink in the upper right of all BMC Defender Server pages.
   • Click the Go To Query hyperlink at the bottom of some search results pages.

   The page provides reasonable default options and retains the values from the previous query that executed.

2. Set the following options:

   Option	Description
   Query Name	(Optional) Query name If you specify a query name, you can access the query later by clicking the Saved Queries hyperlink at the top of the Query page. If you do not specify a query name, the query is not saved, but the results are retained in the query history.
   Query File Type	Type of files to search Each value searches a different area of BMC Defender Server. For an explanation of the different query file types, see Query file types, later in this topic.
   Query Start Date	For most query file types, the start day for the query If you selected External as the Query File Type, select a Microsoft Windows file name.
   Query Span Days	For most query file types, the range of days you want to query For example, if the start date is 2014-01-31, and the Query Span Days is set to one day, then only messages on the selected date are scanned. If you selected External as the Query File Type, select the number of files to span.
   Screen Auto-Refresh	How often you want the screen to refresh while the query is running Each time the screen refreshes, the latest results, if any, are displayed. The value does not affect the update of the background process or the status line indicating the progress of the background process. The default is fifteen seconds, meaning new results are shown every fifteen seconds.
   Max Results	Maximum number of results you want the query to display The default value is fifty matches. The default setting is useful to quickly show results on systems where a large number of matches might exist for the Match Expression (explained later in this topic). When the specified number of results is achieved, the query tool terminates normally.
   Query Seek Order	Order in which the message data is searched: Newer to Older (the default) or Older to Newer The setting can be significant because the Max Results limits the number of matches. If the Max Results is set to fifty, for example, and the Query Seek Order is set to Newer to Older, then the 50 most recent matches are listed. Conversely, if set to Older to Newer, then the 50 oldest matches are listed. This also affects the Trigger Expression (if used).
   Additional Match Qualifiers	Match qualifiers that can precisely target a message, such as matching facility and severities or a specific times of day For more information, see Additional match qualifierslater in this topic.
   Match IP Addr / Group	Range of messages matching a specific IP address or wildcard For example, if you had defined the @@windows_boxes@@ address group on the Correlation > Config > Address Groups page, then you can specify the @@windows_boxes@@ address group.
   Match Expression	Expression for which each message is compared and messages that match are listed as results The expression can be a keyword, wildcard, or logical combination of keywords and wildcards that are potentially parenthetically nested. The match expressions are identical to those found on the Correlation > Threads screen, with the exception that macros are not allowed as part of the expression.

3. To execute the search, click Confirm at the top of the Run Forensic Search Query page.

Additional match qualifiers

On the Query screen, click the Additional Match Qualifiers hyperlink to include one or more of the following fields as part of the search criteria:

• Match Start / End Time—Specify the start and end times of the search in HH : MM : SS format
  The messages returned are delimited as after the match start and before the match end times, inclusive.

• Match Facility / Severity—Specify the match facility and match severity
  You can express the match severity as a range of severities.

• Match Trigger Expression—Find an initial match expression and the search begins after the match expression is located in the log file
  For more information, see Additional notes, later in this topic.

Query file types

The Query screen operates on different data sources, selected using the Query File Type menu at the top of the screen. Various types of file types are supported, as follows:

Query file type	Description
LogFiles	This selection causes the tool to operate on all the messages in the BMC Defender Server\logs folder, that contains a current list of all the messages received during the Keep Days interval (by default 30 days.) When LogFile is selected as the Query File Type, the Query tool operates like the Messages > Search function, except that the search takes place as a background process and the search can use complex match expressions.
Thread-Catalog	This selection causes the tool to operate on a user-selected thread, appearing in the Correlation > Threads screen. This might be the fastest way to run a query (given that the messages being queried all reside in a single defined thread on the system.
User-Catalog	This selection causes the tool to operate on a managed user name, appearing in the Messages > Catalogs > Users screen. The operator specifies the name of a valid user on the Query screen.
Device-Catalog	This selection causes the tool to operate on a managed device name. The operator specifies the name of a valid device exactly as it appears on the Messages > Catalogs > Devices screen.
Archives	This selection causes the tool to operate on all the messages contained in the gzipped archives, residing in the BMC Defender Server\archive folder. When Archive is selected as the Query File Type, the Query tool searches all the archives on the system for the specified message. This can potentially take a long time, even more than a day, given that the BMC Defender Server archives can potentially contain a thousand terabytes or more of message data.
AuxFiles	This selection causes the tool to operate on the Aux files of the system, which is the filtered data. This data is also searchable using the Messages > Aux screen; however, the Query tool performs a more complete job of searching this data and can use complex match expressions to locate specific messages in these files. In this case, the Span Files setting spans the Aux files (and not the days, since Aux files are always deleted at midnight.)
Tickets	This selection causes the tool to operate on Tickets in the system. The data is also searchable using the Advanced Ticket Search screen on the top-level Tickets screen. The setting herein provides an alternate method, including the searching of archived tickets on the system.
External	This selection changes the mode of operation of the Query tool.  Rather than searching message data, the tool simply searches the .log, and .txt files of an external directory. By default, this is the directory is the BMC Defender Server\external folder, but the administrator can change this folder using the Message > Config > Parms screen to be any folder on the system, including shared drives. This function expands the role of BMC Defender Server to include non-message data. In this case, the Span Files setting spans the external file names (and not the days.) The Match IP Addr / Group input is not available for this type of file.
Replay	This selection is similar to external query type but is intended to operate on BMC Defender log files (not arbitrary data, as is the case with the External setting.) The Replay function permits the operator to review historical data associated with BMC Defender log files, such as to support analysts tasked with recreating or investigating long past events or reviewing data that is not part of the BMC Defender system. Specifically, this selection operates on .log files placed in the external directory, as described earlier in this topic. The files must be generated by some copy of BMC Defender Server, and therefore contain the date, time, device, facility, severity, and message content.
BigData-Logs	This search is similar to the LogFiles search but uses a different search methodology which allow large volumes of Log data to be quickly searched. BigData-Logs searches the Log files on the system for the selected date range. Fewer search options are shown compared to other Query File Type searches. The Match Expression supports Regular Expressions which allows for very complex searches.
BigData-Archives	This search is similar to the Archives search but uses a different search methodology which allow large volumes of Log data to be quickly searched. BigData-Archives searches the Log files on the system for the selected date range. Fewer search options are shown compared to other Query File Type searches. The Match Expression supports Regular Expressions which allows for very complex searches.

Additional match qualifiers and trigger expressions

If the operator clicks on the Additional Match Qualifiers hyperlink, more match expressions are added to the screen, which allows the user to first find a pattern before the main search begins.

The Trigger Expression allows the user to search for messages within a specific context of a previous message. For example, the user may wish to find all messages associated with login failures that have been preceded by a specific connection to a VPN. The program will first find the Trigger expression, and then find all messages that follow.

The Trigger Expression is any valid match expression, in a format identical to the main match expression. When a Trigger Expression is used, search results are limited to the same day as the trigger expression (i.e. the Query tool does not span multiple days.)

Additional notes

The amount of time for the query tool to complete depends upon a variety of factors. If the operator is searching for a rarely occurring (or non-occurring) message across all log data or archive data on the system, the query might take ten minutes or more to complete. Conversely, searching for a common message across a limited number of files might return results with a second or two.

The default LogFile search is the slowest but most rigorous way of extracting data from the system, and scans all log files on the system for data; this is rigorous but EXTREMELY SLOW. A better way of performing a query is to search a particular Correlation Thread, which can be 1,000 times faster than simply searching all log data from beginning to end.

You can save a query by specifying a name, and then reuse that query by clicking the Saved Queries hyperlink at the top of the Query page. Your last ten query results are saved and available for review and selection using the History link.