System requirements for a dedicated Remote ETL Engine and Gateway Server

Use the following hardware requirements and supported operating systems for a dedicated Remote ETL Engine and Gateway Server. 

When using the Gateway Server, you must install it on the same computer where the Remote ETL Engine is installed. We recommend using a dedicated Remote ETL Engine for processing data from the Gateway Server and a separate Remote ETL Engine for processing data from other on-premises ETLs. The use of separate Remote ETL Engines ensures efficient data processing.

Ensure that the Linux computer where you plan to install the Remote ETL Engine and Gateway Server meets the following requirements:

Hardware requirements

  • CPU - 2 processors
  • RAM - 8 GB
  • Disk - 218 GB

Supported operating systems

The computer that hosts the Remote ETL Engine and Gateway Server must be running one of the following operating systems. Only x86_64 architecture is supported for the operating systems.

Operating system

Version

Required libraries
Rocky Linux8.5, 8.6, 8.7, 8.8ksh, glibc.i686, perl-5.16.3, libXtst, liberation-sans-fonts, libXrender
Red Hat Enterprise Linux19.0 and laterksh, glibc.i686, perl-5.16.3, libXtst, liberation-sans-fonts, libXrender
8.0 and laterksh, glibc.i686, perl-5.16.3, libXtst, liberation-sans-fonts, libXrender
7.6 and laterksh, glibc.i686, perl-5.16.3, libXtst, liberation-sans-fonts, libXrender
Oracle Linux7.6 and laterksh, glibc.i686, perl-5.16.3, libXtst, liberation-sans-fonts
CentOS7.6 and laterksh, glibc.i686, perl-5.16.3, libXtst, liberation-sans-fonts

SUSE Linux Enterprise Server

12 SP4 and later

ksh, glibc.i686, perl-libwww-perl


15 SP1 and later

ksh, glibc.i686, perl-libwww-perl

For Linux Gateway Servers and Linux Continuous Optimization Agents, systems running kernel versions up to and including 5.x are supported.

1The ksh package is required and is not installed by default. Unavailability of this package might cause installation or run time issues. Use the following commands to check if the package is installed:

$ rpm -qa | grep -E 'ksh'
ksh-1.0.0~beta.1-2.el9.x86_64

Ensure that you install the package as a root user. Use the following command to install the package: 

yum install -y ksh

Other installation requirements

  • Install the Gateway Server on the same computer where you have installed the Remote ETL Engine.
  • Use a non-privileged and non-root user account for installing the Gateway Server. The non-privileged user must be created on the installation computer before running the installer and must have access to use cron on the computer. This user owns all the Gateway Server files and runs most of the processes. The installer does not create the installation owner.
  • The available temporary disk space must be greater than 500 MB. The Gateway Server installer uses the following environment variables in the listed order to access this space:
    • $IATEMPDIR environment variable
    • /tmp
    • Your home directory
  • On Linux systems, ensure that at least 2 MB of free space is reserved for the /etc file system on all the managed systems, and you have the execute (x) permission for the /etc directory.
  • Make sure that TCP/IP is installed on the Gateway Server computer and the managed systems that run Capacity Agents. The Gateway Server uses the TCP/IP protocol to communicate with these managed systems.
  • Install the pcron utility that is required to schedule Manager runs simultaneously.
  • To use remote data repositories on a UNIX network file system, make sure that the rpc.lockd and rpc.statd NFS lock manager daemons run on both the client and server computers.

  • The ksh library must be available on the Linux system where you want to install the Gateway Server.
  • For Linux systems, ensure that the installation directory is on a standard Linux file system such as ext3 or a Global File System (GFS) for high availability deployments (Active/Passive servers). Installation on a CIFS share mounted as a file system is not supported. You can use CIFS share only for the shared repository.

Sizing and scalability guidelines

Review the following guidelines to estimate the required disk, memory, and processor capacity.

Remote ETL Engine

ETL Engine servers can be scaled horizontally and vertically. The major sizing drivers for ETL Engine servers are:

  • The required data processing throughput in samples per day. This value is the multiplication of number of managed entities and the average number of samples collected for each entity in a day.
  • The number of connector instances (tasks) scheduled on the ETL Engine.

Disk space guidelines

The default values allow for ten days of temporary files and log files accumulated during the normal day-to-day population activity. The default period for the File System Cleaner system task is ten days. If you increase this period for any reason, adjust the sizing and scalability numbers accordingly.

You need additional disk space for special activities:

  • Bulk import of data, for example, for addition of new data sources with historical data.
  • Recovery import of data when a data source stops for a day or two for any reason and has to be recovered.

For these special activities, estimate additional capacity using the number of anticipated additional samples per day. Temporary files and logs from these samples will remain on the disk for ten days (or whatever the File System Cleaner system task period is set to).

Gateway Server

The following factors affect the sizing drivers:

  • Size of your environment
  • Number of Agents from which data is collected
  • Data retention period

Data retention

The data collected by Continuous Optimization Agents (in the UDR format) is periodically transferred to the Gateway Server where it is automatically processed into hourly intervals and saved in text files called VIS files. We recommend the following guidelines for the retention period of UDR and VIS files:

  • Retain the VIS files of earlier 3 months. These files are useful if you need to recover data. For details, see Recovering data.
  • For UDR and VIS files, keep 3 months of data. Out of this data, the oldest two months data can be compressed to save space.

The following table provides the recommendations for the hardware capacity according to the environment size.

Size of your environmentNumber of servers

Processor cores per server

RAM (4GB/core) per server

IOPS per serverNumber of tasks per serverStorage (in GB) per server
InstallationRemote ETL EngineGateway Server

Small

(up to 1000 servers)

1286020850160

Medium

(up to 5000 servers)

2416150408100395

Large

(up to 10000 servers)

3416200408100525

The calculations for the Gateway Server storage requirements are based on the following assumptions:

  • Metric resolution - 60 minutes
  • Processing window - 4 hours
  • UDR data retention - 1 month
  • VIS data retention - 3 months
  • UDR spill interval - 15 minutes
Was this page helpful? Yes No Submitting... Thank you

Comments