End-to-end process


If you are a new user, use this topic to understand the basic end-to-end process of using the product (with an example).

To get a high-level understanding of the end-to-end process, see the conceptual workflow at Using.

The following sections will guide you through the process based on an example scenario:

Scenario for using IT Data Analytics

Suppose an application has a special user who is required to administer the application. This user has additional privileges that he can use for placing orders.

However, this user must not be used for placing any order using the application.

Suppose you want to be notified each time this user tries to place an order.

Before you begin

Ensure that you have already downloaded and installed the product. For more information, see Installing.

Step 1: Locate and analyze the data

This step is applicable to administrators only.

When you start using IT Data Analytics, you need to first locate and analyze the data that you want to collect.

The following table provides sample data that you can collect and index.

Sample data

16 Feb 2017 10:58:28 [DEBUG] LoginService - [Thread=Server(77)] 10.20.30.40 - Login request from user=Mike
16 Feb 2017 10:58:28 [DEBUG] LoginService - [Thread=Server(77)] 10.20.30.40 - some other message user=Mike sessionid=1234
16 Feb 2017 10:58:28 [DEBUG] LoginService - [Thread=Server(77)] 10.20.30.40 - sessionid=1234 add product=PenDrive to cart price=123
16 Feb 2017 10:58:28 [DEBUG] LoginService - [Thread=Server(77)] 10.20.30.40 - sessionid=1234 add product=Phone to cart price=345
16 Feb 2017 10:58:28 [DEBUG] LoginService - [Thread=Server(77)] 10.20.30.40 - sessionid=1234 place order
16 Feb 2017 10:58:30 [DEBUG] LoginService - [Thread=Server(77)] 10.20.30.40 - sessionid=1234 logout

Step 2: Identify a data pattern for indexing the data

To perform this step, you need to log on to the product with app admin or super admin credentials.

The next step is to identify a data pattern that captures the pattern in the sample data and extracts fields that can be useful while searching. The product provides a list of default data patterns that you can directly use for collecting data. You need to see if one of the existing data patterns match the data that you want to collect. Otherwise, you can create a new data pattern. For more information, see Setting-up-data-patterns-to-extract-fields.

In this case, you need to create a data pattern for indexing the sample data by navigating to Administration > Data Patterns. While creating the data pattern, select the matching date format and directly skip to the final step of saving the data pattern. For more information, see Creating-data-patterns. Thereafter, edit the data pattern and use the details provided in the following table to edit the primary pattern and extract fields. For more information, see Editing-or-cloning-data-patterns.

The following table guides you through the high-level process of creating the data pattern by using the wizard and then editing the data pattern to extract fields:The [confluence_table-plus] macro is a standalone macro and it cannot be used inline. Click on this message for details.

Step 3: Create a data collector by using the data pattern identified in the earlier step

To perform this step, you need to log on to the product with app admin or super admin credentials.

Create a data collector based on your environment and where your data resides. For example, to collect files locally, you need to create the Monitor File on Collection Agent data collector.

For more information, see the following links:

In this scenario, you can create a data collector of the Upload File type (as shown in the following figure). For more information, see Collecting-data-from-an-individual-file.

collectorcreation.png

During the data collector creation, you can optionally specify tags to enable effective searching. The following table provides a list of sample tags.

Sample tags

Tag

Valuet

os

Linux

tier

application

appgroup

myapp

Step 4: Perform a search

After data collection is complete, you can search the data in various ways, for example, by using fields, tags, and search commands.

For more information, see the following topics:

The following table provides a list of sample search queries that you can use to search the data.

Sample search strings

What to do?

Search string

Search the entire application, across all tiers

appgroup=myapp

Find all activity of users and their sessions

appgroup=myapp | group user,sessionid

Find users who lost interest quickly

appgroup=myapp | group user,sessionid | filter greaterthan(duration, "30")

Find activity of all special users and their sessions

appgroup=myapp | group user,sessionid | filter match(user, "special_user")

Step 5: Create a saved search

If you want to monitor the data collected, you need to create a saved search. Saved searches an help you save important search queries that you might want to reuse in the future. Also, saved searches are the building blocks for creating dashboards and notifications. For more information, see Saving-and-sharing-searches-for-analytics-and-monitoring.

In this scenario, use the following sample search query to create a saved search:

appgroup=myapp | group user,sessionid | filter match(user, "special_user")

Step 6: Create a notification based on the saved search

To perform this step, you need to log on to the product with app admin or super admin credentials.

Notifications can help you monitor the data collected based on certain conditions. For more information, see Setting-up-notifications-to-create-alerts-or-reports.

In this scenario, create a notification based on the saved search that you created in the previous step.

Use the following inputs while creating the notification:

  • Condition count: > 0
  • Email notification to: admin@acme.com

This is the final step by which you can be notified if the special user tries to place an order.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*