Estimating the disk space on the Agent system for historical data

Before starting data collection, estimate the disk space that is required for storing historical data on the computer where the Gateway Server is installed. If you configure data collection at one second granularity, the size of the historical data grows rapidly, which requires additional disk space. Use this information for estimating the disk size (in bytes) that is required for historical data.

To calculate the disk size for the default agent history configuration

The default configuration stores 49 hours of historical data that the Agent collects. This default period is specified in the Repository.cfg file located at the following path:

  • UNIX: $BEST1_HOME/bgs/mgroup/
  • Windows: %BEST1_COLLECT_HOME%\bgs\mgroup

Use the following formula to approximately calculate the disk size that is required for storing this historical data. This formula applies to both UNIX and Windows systems.

Disk size required (in bytes) = 750000(Directory structure size) + nIntervals [36(nCPUs) + 100(nDisks) + (200)nFileSystems + 100(nNetworkInterfaces) + (nProcesses)(400 + CommandSize)]

where

  • Directory structure size: Size of the directory structure. For UNIX, this value is 750,000, and for Windows the value is 850,000.
  • nIntervals: Number of intervals that depend on the parameters in the repository.cfg file. For more information, see number of intervals.
  • nCPUs: Number of CPUs
  • nDisks: Number of disks recognized by the kernel
  • nFileSystems: Number of file systems
  • nNetworkInterfaces: Number of network interfaces
  • nProcesses: Number of processes in the system process table. The average process size is 400 bytes.
  • CommandSize: Average command size that is determined by the average size of the command string, which is 256.

To calculate the number of intervals

Calculate the number of intervals by adding specific parameters along with their corresponding values to the Repository.cfg configuration file. You can also edit the parameter values in this file according to the interval requirement.

Example

HISTORY
      SAMPLING_INTERVAL = 10
     LENGTH = 60
      SUMMARIZATION
         SUMMARIZATION_INTERVAL = 5
        LENGTH = 480
         SUMMARIZATION
            SUMMARIZATION_INTERVAL = 15
           LENGTH = 2400
         END_SUMMARIZATION
      END_SUMMARIZATION
    END_HISTORY

The syntax in the preceding code example is explained as follows:

 Example

To estimate the disk space for the following Windows system:

The required disk space will be:

850,000 + (616)[(36(2) + 100(4) + 200(14) + 100(1) + (240)(400 + 256)] = 850,000 + 616 (160,812)
= 99,910,192 (about 100 MB)

To calculate the additional disk size required for one-second data collection

If you configure one-second data collection for specific metric groups, you need additional disk space.

Example

You have configured one-second data collection for the following metric groups: Processor Statistics, Disk Statistics, File System Statistics, Network Interface Statistics, and Process Statistics.

For these metric groups, you want to collect one-second historical data for 24 hours. The total number of intervals will be:
3600 x 24 (3600 intervals per hour x number of hours in a day) = 86,400

Calculate the additional disk space as follows:

Metric group

Number of records

Additional disk space required

Processor Statistics

4

Bytes per record x nCPUs x nIntervals
(36 x 4 x 86,400)

Disk Statistics

12

Bytes per record x nDisks x nIntervals
(100 x 12 x 86,400)

File System Statistics

16

Bytes per record x nFileSystems x nIntervals
(200 x 16 x 86,400)

Network Interface Statistics

6

Bytes per record x nInterfaces x nIntervals
(100 x 6 x 86,400)

Process Statistics

250

Bytes per record x nProcesses x nIntervals
((400 + CommandSize) x 250 x 86,000)

To limit the amount of data collected for the NT User Configuration metrics

Data collection in Microsoft Windows for the domain controller causes the NT User Configuration metrics to be extremely large in size, which can have the following effect:

  • High CPU utilization by the bgscollect process on the computer
  • Creation of large NT User Configuration UDR data files

  1. Modify (or create, if one does not exist) the Collect.cfg configuration file so that it matches the following syntax:

    BEGIN_CONFIG
       BEGIN_NT_SYSTEM_COLLECT
         COLLECT_ALL_USERGROUP = FALSE
        MAX_USER_RECORDS = 500
       END_NT_SYSTEM_COLLECT
     END_CONFIG
  2. Restart the Agent on the remote computer.

This modified setting instructs the Collector to limit data collection to 500 user records on the computer. For more information about the Collect.cfg file, see Collect configuration file.

Where to go from here

Install the remote components to collect on-premises data