Page tree

Product performance is impacted by various factors, depending on your deployment size and scale. You can use the following information as a high-level guide for understanding the factors that can impact your deployment sizing and the approximate impact they are likely to have on product performance.

Variables that impact product performance

This section provides information about the variables that can impact the product performance along with the levels of impact to each of the system resources, as follows:

Variables that impact product performance

VariableImpact on resources 
Processor1Memory2Storage3I/O4Comments
Number of data collectorsMediumLowMediumMedium 
Retention periodNoneMediumHighLow 
Number of tags defined for the data collectorsMediumLowMediumLow 
Number of fields added to the My Fields panel on the Search tabHighNoneNoneLow 
Number of fields specified in data patternsLowNoneMediumNone 
Number of unique field values in the My Fields panel on the Search tabLowHighHighHigh 
Volume of data indexed per dayHighLowHighHigh 
Number of managed hostsLowNoneNoneNone 
Simple search commands
(Search commands with nontabular output)
LowLowNoneLow 
Advanced search commands
(timechart, stats, top, rare, and group)
MediumMediumNoneMedium

Requires high CPU utilization by the Indexer and Search components.

Avoid using high cardinality fields (fields with large number of unique values) in the by parameters. This is applicable while running those search commands where the by parameter is supported.

Number of concurrent usersHighHighNoneHigh 
Number of notification alertsLowLowNoneLowNotification alerts are meant for shorter time period searches.
Number of notification reportsLowLowNoneLowNotification reports are expected to range for medium time period searches.
Number of viewlets and viewsMediumMediumNoneMediumPerformance overhead of a viewlet is the same as that of a manual run of the saved search used for creating the viewlet.

1 Processor utilization (CPU)

2 Memory utilization (RAM)

3 Storage utilization (disk space)

4 I/O transfer rate (disk read/write rate)

Sizing drivers and their impact

Use the information in this section to calculate the deployment size of the product in your environment.

Note

This information is indicative and is available for reference purposes.

The following table describes some of the primary drivers that impact product performance, along with their approximate impact on sizing:

DriversConsiderations and impact

Retention period

An increase in the retention period impacts the memory and storage requirements for the Indexer.

Impact on memory

The product stores data in time-based indices (buckets). The capacity of each bucket can be calculated in terms of either volume of data or time duration (in hours).

For each index, the minimum amount of RAM required by the Indexer is approximately 21 MB.

By default, each day, data is indexed into 4 buckets of 6 hours each. For a 7-day retention period (the default), you need 4 (buckets) * 7 (retention in days) = 28 buckets.

The total RAM requirement = 28 (buckets) * 21 (MB per bucket) = 588 MB.

  • RAM required (in MB) = Number of buckets required * 21

  • Number of buckets required = 4 (configurable) * Retention period (in days)

Impact on storage

For every 1-day increase in your retention period, you require an increase of 1.25 times the volume of incoming data (per day).

With a 7-day retention period (the default), if you are indexing 10 GB of data per day, your storage requirement can be calculated as 7 (retention in days) * 10 (incoming data in GB) * 1.25 = 87.5 GB

Storage required = Retention period (in days) * Volume of data (per day) * 1.25

Number of concurrent users

Multiple threads are created for every search that is triggered. The time taken by each thread is minimal and is approximately an average of 0.5 CPU cores per user (for each search execution).

Each notification or viewlet execution is the equivalent of one search execution.

Impact on CPU

Parallel executions of search requests, viewlets, and notifications by concurrent users can impact the CPU utilization of your system. This factor is applicable to the Indexer and Search components.

CPU required = (Number of notification executions + Number of viewlet executions + Number of search executions) * 0.5

Impact on memory

For every search that you execute, a certain amount of data is loaded to the memory, especially for fields (on the My Fields panel on the Search tab) and advanced search commands (for example, group and stats). The memory impact is higher if the fields involved are high cardinality fields with many unique values. The Java heap size requirement increases in proportion to the number of searches performed by concurrent users.

For example, eight concurrent searches running field calculations for a high cardinality field against the same index at the same time will most likely require the same Java heap size as that of one field calculation. On the other hand, eight concurrent searches running field calculations for a high cardinality field against different indices at the same time requires around eight times the Java heap size of one field calculation.

Fields impact the Indexer memory, while advanced search commands impact the Console server memory.

No simple formula exists for calculating the impact of the number of concurrent users on memory.

Volume of data indexed (per day)

The volume of data indexed per day can impact your storage requirements.

Impact on memory

The product stores data in time-based indices (buckets). The capacity of each bucket can be calculated in terms of either volume of data or time duration (in hours). Every search performed has an impact on the memory of the Indexer. The bucket capacity plays an important role in helping you control the memory needs.

No index can contain more than 5 GB of data. If you want to index 20 GB of data per day, then 20 / 5 = 4 buckets per day.

Therefore, the indexed duration (in hours) is 24 / 4 = 6 hours. This value must be used as the value for the intervalInHrs property. For more information, see Component configuration recommendations.

Number of buckets required (per day) =  Volume of data (per day) / 5

Indexed duration (in hours) = 24 / Number of buckets required (per day)

Impact on storage

The storage required for your system is 1.25 times the volume of data (indexed per day) depending on the defined data-retention period.

Storage required =  1.25 * Volume of data (per day) * Retention period (in days)

Related topic

 FAQs about performance and scalability