Estimating the disk space on the Agent system for historical data

Before starting data collection, you must estimate the disk space that is required for storing historical data on the computer where the Gateway Server is installed. If you configure data collection at one second granularity, the size of the historical data grows rapidly, which requires additional disk space. This topic provides information about approximately estimating the disk size (in bytes) that is required for historical data.

Calculating the disk size for the default agent history configuration

The default configuration is to store 49 hours of historical data that the Agent collects. This is the default period that is specified in the Repository.cfg file located at the following path:

• UNIX: $BEST1_HOME/bgs/mgroup/
• Windows: %BEST1_COLLECT_HOME%\bgs\mgroup

Use the following formula to approximately calculate the disk size that is required for storing this historical data. This formula is applicable to both UNIX and Windows systems.

Disk size required (in bytes) = 750000(Directory structure size) + nIntervals [36(nCPUs) + 100(nDisks) + (200)nFileSystems + 100(nNetworkInterfaces) + (nProcesses)(400 + CommandSize)]

where, 

• Directory structure size: The size of the directory structure. For UNIX, this value is 750000, and for Windows the value is 850,000
• nIntervals: Number of intervals that depend on the parameters in the repository.cfg file. For more information, see number of intervals.
• nCPUs: Number of CPUs
• nDisks: Number of disks recognized by the kernel
• nFileSystems: Number of file systems
• nNetworkInterfaces: Number of network interfaces
• nProcesses: Number of processes in the system process table. The average process size is 400 bytes.
• CommandSize: Average command size that is determined by the average size of the command string, which is 256.

Number of intervals

The number of intervals are calculated by adding specific parameters along with their corresponding values to the Repository.cfg configuration file. The parameter values in this file can be edited according to the interval requirement.

Consider the following example where a portion of code from the configuration file is listed:

HISTORY
      SAMPLING_INTERVAL = 10
      LENGTH = 60
      SUMMARIZATION
         SUMMARIZATION_INTERVAL = 5
         LENGTH = 480
         SUMMARIZATION
            SUMMARIZATION_INTERVAL = 15
            LENGTH = 2400
         END_SUMMARIZATION
      END_SUMMARIZATION
    END_HISTORY

The syntax in the preceding code example is explained as follows:

 Back to top ↑

Example
You want to estimate the disk space for the following Windows system:

The required disk space will be:

850,000 + (616)[(36(2) + 100(4) + 200(14) + 100(1) + (240)(400 + 256)] = 850,000 + 616 (160812) 
= 99,910,192 (about 100 MB)

Back to top ↑

Calculating the additional disk size for one second data collection

If you configure one second data collection for specific metric groups, you need the additional disk space.

Example
You have configured one second data collection for the following metric groups: Processor Statistics, Disk Statistics, File System Statistics, Network Interface Statistics, Process Statistics

For these metric groups, you want to collect one second historical data for 24 hours. The total number of intervals will be:
3600 * 24 (3600 intervals per hour * number of hours in a day) = 86,400

Calculate the additional disk space as follows:

Back to top ↑

Impact of data collection on NT User Configuration metrics

Data collection in Microsoft Windows for the domain controller causes the NT User Configuration metrics to be extremely large in size, which can have the following effect:

• High CPU utilization by the bgscollect process on the computer
• Creation of large NT User Configuration UDR data files

To control the amount of data collected for this metric group

1. Modify (or create, if one does not exist) the Collect.cfg configuration file so that it matches the following syntax:

   BEGIN_CONFIG
      BEGIN_NT_SYSTEM_COLLECT
        COLLECT_ALL_USERGROUP = FALSE
       MAX_USER_RECORDS = 500
      END_NT_SYSTEM_COLLECT
    END_CONFIG

2. Restart the Agent on the remote computer.

This modified setting instructs the Collector to limit data collection to 500 user records on the computer. For more information about the Collect.cfg file, see Collect configuration file.

Back to top ↑

Control the size of Investigate history

You can view Investigate history from the Gateway Server through the Investigate GUI. The default amount of Investigate history saved on the remote agent node is approximately 48 hours. You can change this value on the remote agent node to retain more or less history data.

The amount of Investigate history is controlled by the Repository.cfg file. To modify the amount of Investigate history retained, refer to the following sections: 

Determining the amount of Investigate history to be retained

When determining the amount of history data to retain, consider the following values:

• number of 10-second sample intervals
• number of 5-minute summarized spills
• number of 15-minute summarized spills

By default, Investigate history retains:

• 1 hour of 10-second raw data samples
•  8 hours of 5-minute summarized spills
• 40 hours of 15-minute summarized spills

These values are determined by the following entries in the Repository.cfg file:

 HISTORY SAMPLING_INTERVAL = 10
 LENGTH = 60 <-- This value is the number of minutes of 10-second samples.
 SUMMARIZATION
 SUMMARIZATION_INTERVAL = 5
 LENGTH = 480 <-- This value is the number of minutes of 5-minute summarized spills.
 SUMMARIZATION
 SUMMARIZATION_INTERVAL = 15
 LENGTH = 2400 <-- This value is the number of minutes of 15-minute summarized spills.
 END_SUMMARIZATION
 END_SUMMARIZATION
 END_HISTORY

Back to top ↑

Modifying Investigate history on a UNIX remote agent node

1. Change to the $BEST1_HOME/bgs/mgroup directory:

   > cd $BEST1_HOME/bgs/mgroup

2. Create a backup copy of the Repository.cfg file:

   > cp Repository.cfg Repository.cfg.sav

3. Using a text editor, modify the following section in the Repository.cfg file to the appropriate values:

   # 10 second sampling interval
    # 1 hour of 10 second raw data
    # 8 hours of five minute summarized spills
    # 40 hours of fifteen minute summarized spills
    # HISTORY SAMPLING_INTERVAL = 10
    LENGTH = 60 <-- This value is the number of minutes of 10-second samples.
    SUMMARIZATION
    SUMMARIZATION_INTERVAL = 5
    LENGTH = 480 <-- This value is the number of minutes of 5-minute summarized spills.
    SUMMARIZATION
    SUMMARIZATION_INTERVAL = 15
    LENGTH = 2400 <-- This value is the number of minutes of 15-minute summarized spills.
    END_SUMMARIZATION
    END_SUMMARIZATION
    END_HISTORY

4. Save the file.

5. Stop the Agent.

   > $BEST1_HOME/bgs/scripts/best1agent_stop -b $BEST1_HOME

6. Restart the Agent.

   > $BEST1_HOME/bgs/scripts/best1collect -B $BEST1_HOME -q

   The updated Repository.cfg file is read by the agent on startup, and the updated summarization interval options are active.

Back to top ↑

Modifying Investigate history on a Windows remote agent node

1. Open Microsoft Windows Explorer and browse to the %BEST1_HOME%\bgs\mgroup directory.
2. Right-click the Repository.cfg file, select Open With, and select a text editor.

3. Modify the following section in the Repository.cfg file to the appropriate values:

   # 10 second sampling interval
    # 1 hour of 10 second raw data
    # 8 hours of five minute summarized spills
    # 40 hours of fifteen minute summarized spills
    # HISTORY
    SAMPLING_INTERVAL = 10
    LENGTH = 60 <-- This value is the number of minutes of 10-second samples.
    SUMMARIZATION
    SUMMARIZATION_INTERVAL = 5
    LENGTH = 480 <-- This value is the number of minutes of 5-minute summarized spills.
    SUMMARIZATION
    SUMMARIZATION_INTERVAL = 15
    LENGTH = 2400 <-- This value is the number of minutes of 15-minute summarized spills.
    END_SUMMARIZATION
    END_SUMMARIZATION
    END_HISTORY

4. Save the file.
5. Stop the Agent by selecting Start > Programs > BMC PATROL > Perform > Stop Agent.
6. Start the Agent by selecting Start > Programs > BMC PATROL > Perform > Collect Data.
7. From a collect process, select Query Information and specify the computer name of the agent computer to restart.
   A collect query is sent to the server, which restarts the bgsagent process.

Back to top ↑