Sizing drivers and their impact

Retention period

An increase in the retention period impacts the memory and storage requirements for one or more Indexers deployed in your environment.

Impact on memory

The product stores data in time-based indices (buckets). The capacity of each bucket can be calculated in terms of either volume of data or time duration (in hours).

For each index, the minimum amount of RAM required by the Indexer is approximately 18 MB.

By default, each day, data is indexed into 4 buckets of 6 hours each. For a 7-day retention period (the default), you need 4 (buckets) * 7 (retention in days) = 28 buckets.

The total RAM requirement = 28 (buckets) * 18 (MB per bucket) = 504 MB.

Impact on storage

For every 1-day increase in your retention period, you require an increase of 1.3 times the volume of incoming data (per day).

With a 7-day retention period (the default), if you are indexing 10 GB of data per day, your storage requirement can be calculated as 7 (retention in days) * 10 (incoming data in GB) * 1.3 = 91 GB

Multiple threads are created for every search that is triggered. The time taken by each thread is minimal and is approximately an average of 0.5 CPUs per user (for each search execution).

Each notification or dashlet execution is the equivalent of one search execution.

Impact on CPU

Parallel executions of search requests, dashlets, and notifications by concurrent users can impact the CPU utilization of your system. This factor is applicable to one or more Indexers and one or more Search components.

Impact on memory

For every search that you execute, a certain amount of data is loaded to the memory, especially for fields (on the Filters panel on the Search tab) and advanced search commands (for example, group and stats). The memory impact is higher if the fields involved are high cardinality fields with many unique values. The Java heap size requirement increases in proportion to the number of searches performed by concurrent users.

For example, eight concurrent searches running field calculations for a high cardinality field against the same index at the same time will most likely require the same Java heap size as that of one field calculation. On the other hand, eight concurrent searches running field calculations for a high cardinality field against different indices at the same time requires around eight times the Java heap size of one field calculation.

No simple formula exists for calculating the impact of the number of concurrent users on memory. You can allocate an additional 100 MB RAM per concurrent user, this additional RAM is required for the Indexer component.