Running a BigData search

The Query Report has query file type options that you can use to search large volumes of log and archive data. With BigData searches, you can search large volumes of data much faster than the other query file type searches.

The BigData query types have fewer options than other query types. The Query Start Date and Query Span Days options (or Archive Start Date and Archive Span Days options) are standard to query types, but the Match Expression option supports different search types:

• You can use complex searches in the form of regular expressions. BigData searches do not support lists or macros, which are available for other query types.
• You can duplicate search criteria from other query-type fields as part of a regular expression query.

To run a BigData search

1. Navigate to the Reports > Query page and click the Generate button to access the Run Forensics Search Query page.
2. (Optional) Enter a Query Name.
   • If you specify a query name, you can access the query later by clicking the Saved Queries hyperlink at the top of the Query page.
   • If you do not specify a query name, the query is not saved, but the results are retained in the query history.
3. From the Query File Type field, select one of the following options: BigData-Logs or BigData-Archives.
   Depending on your selection, one of the following pages is displayed:
   
   • BigData-Logs
     
     
   • BigData-Archives
     
4. Set the query or archive start date and span days.
5. In the Match Expression box, enter a regular expression.
   Use caution when searching large amounts of data for generic or common terms because it is very easy to generate result files that are several gigabytes in size.
6. To execute the search, click Confirm at the top of the Run Forensic Search Query page.

When you run a BigData query, all results are returned. Unless you terminate the query, it runs until all the data is searched and all the results are returned.

Log files are searched directly but archives must first be uncompressed before they are searched. The BigData searches do not have the same time, size, or line limitations that affect other searches. For long-running searches, the results page automatically refreshes periodically. New results are visible as they are found in approximately five-second intervals. When the search is complete, the results are available and searchable, just like for other query types.

Additional notes

• Searching files on a solid-state drive (SSD) device has significant advantages over a standard hard drive device. The BigData search method is very disk intensive and maximizes the read/write speed while running.
• Searches that return fewer results return faster, while result files that take up several gigabytes increase the search time.