Installing remote components to collect on-premises data

For information about system requirements and sizing guidelines, see Planning.

This section describes how to install the Remote ETL Engine silently. 

The following video (4:16) illustrates the process of installing the Remote ETL Engine.

https://youtu.be/US471Al8zuk

A sample response file with default settings is included in the Remote ETL Engine install image. Modify the sample file instead of creating a new file. The sample file also includes instructions on how to run the installation silently.

1. Log in as a root user to the computer where you want to install the Remote ETL Engine, and create a temporary directory. Example: HOCO_temp.

2. Download the installation files from ftp://<customerprefix>-opt-ftp.onbmc.com/FromOnDemand/optimize-onprem.
   

3. Extract the downloaded files to the temporary directory. Ensure that the temporary directory is different from the installation directory to avoid any file sharing conflict.

4. Generate the API key from the Helix Capacity Optimization Console. You will need to specify this key while configuring the bearer authorization token in the options file.

   1. Log in to the Helix Capacity Optimization Console as ho-admin or other admin user.

   2. Select Administration > Users > Roles.
      The Roles page shows a summary table listing the currently defined user roles, their description, and the associated Helix SSO (local or integrated LDAP) groups as external names, if applicable.

   3. Click the ADMIN role name. 
      The detail page for the selected role is displayed in the working area, listing all activities assigned to the role.

   4. Click Generate API key. 
      
   5. On the Generate API key page, select No expiration date.
      The expiration date is the date when the API key will no longer be useful for authentication. The default expiration date is one month from the date of the API key generation.

   6. Click Generate. The credentials are encrypted and downloaded as the credentials.key file.

   7. Open the credentials.key file in a text editor.

      Contents of the credentials.key file

      The credentials.key file contains the following:

      • COConsoleURL: The URL of the Helix Capacity Optimization Console from where you generated the API key.

      • Authorization: The authorization key. The token includes the role ID that generated the key, activities assigned to the role at the time of creation, and the expiration date of the key. After the key is generated, if there are any changes to the activities assigned to the role that generated the key, those changes will not be reflected in the already generated key. You can use this authorization key to make authenticated calls to any API. For more information, see Accessing the public APIs.

        Example of the credentials.key file

        {"COConsoleURL":"https://ho-ref-prod-optconsole.bmc.com","Authorization":"Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJBRE1JTiIsInJvbGVfaWQiOjAsInJvbGVzIjpbIldFQl9DT05GSUdVUkVfQUxMIiwiV0VQ"}

        
        

   8. Copy and paste the authorization token from generated key to the silent installation response file (BCO_ADDITIONAL_SERVER_REMOTE_EE.txt) on the Remote ETL Engine in the following format:
      -J AUTH_TOKEN=Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJBRE1JTiIsInJvbGVfaWQiOjAsInJvbGVzIjpbIldFQl9DT05GSUdVUkVfQUxMIiwiV0VQ

      The token must be on a single line, including the word Bearer.

5. At the shell prompt, change to the directory where you extracted the downloaded files. Example: HOCO_temp/BCO/Disk1.

6. Navigate to the directory where you extracted the downloaded files and locate the sample response file. Example: HOCO_temp/BCO/Disk1/silentInstallTemplates/BCO_ADDITIONAL_SERVER_REMOTE_EE.txt.

7. Edit the BCO_ADDITIONAL_SERVER_REMOTE_EE.txt file in a text editor.

8. In the -J AUTH_TOKEN property, paste the token that you have copied in Step 4. Modify other properties of the file according to your preferences as listed below, and save the file.

    Property details

   Prefix tags for the silent installation properties:

   • -P: Precedes the directory properties

   • -A: Precedes the application features
     

   • -J: Precedes the Java properties

   Property name	Description
   Installation settings
   -P installLocation	Installation directory path. Default value: /opt/bmc/BCO Example: -P installLocation=/opt/bmc/BCO
   For root user
   -J BCO_USER	Name of the user (new or existing). Default value: cpit Example: -J BCO_USER=cpit
   Helix host details
   -J URL_HOST	External data API server name. Example: -J URL_HOST=companyname-optapi.onbmc.com
   Authorization Bearer Token
   -J AUTH_TOKEN	Bearer authorization key. This key is used to authenticate the user when connecting to the Helix Capacity Optimization Console. For instructions about generating the key, see Step 3. Example: -J AUTH_TOKEN=Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJBRE1JTiIsInJvbGVfaWQiOjAsInJvbGVzIjpbIldFQl9DT05GSUdVUkVfQUxMIiwiV0VQ
   Proxy server connection
   -J IS_PROXY_CONFIGURED	Configure the proxy server. Default value: false Example: -J IS_PROXY_CONFIGURED=false
   -J REE_PROXY_HOST	Name of the proxy server host. -J REE_PROXY_HOST=<proxy_server_hostname> Example: -J REE_PROXY_HOST=vw-abc-dev01.bmc.com
   -J REE_PROXY_PORT	Port number to communicate with the proxy server. Example: -J REE_PROXY_PORT=8090
   -J IS_PROXY_AUTHENTICATION_REQUIRED	If the proxy server requires authentication, set this value to true and specify the proxy server user name and password. Default value: false Example: -J IS_PROXY_AUTHENTICATION_REQUIRED=false
   -J REE_PROXY_USER	Proxy server user name. Example: -J REE_PROXY_USER=proxy_user
   -J REE_PROXY_SECPWD	Proxy server password. Example: -J REE_PROXY_SECPWD=proxy_user_password
   Advanced properties
   -J IS_RECONFIGURATION_FLOW -J IS_REPAIR_FLOW	To run the installer on the existing installation, choose one of the following modes. You can set only one of these properties to true as the installer can run only in one of these modes. Reconfigure: Use the reconfigure mode to modify certain values that you had provided during installation, such as database connection or repository option, in the installed product.  -J IS_RECONFIGURATION_FLOW Default value: false Re-installation: Use the re-installation mode to re-install the Application Server or ETL Engine. BMC recommends that you back up your database before starting the re-installation. -J IS_REPAIR_FLOW Default value: false
   Do not modify these settings
   -J INSTALLATION_TYPE_SELECTION	Retain the default installation type. Default value: EXTERNAL_SERVER Example: -J INSTALLATION_TYPE_SELECTION=EXTERNAL_SERVER
   -A <feature name>	Install the Remote ETL Engine component. Default value: featureCoreETL Example: -A featureCoreETL
   -J IS_REMOTE_COMPONENT	Retain the default value. Default value: true Example: -J IS_REMOTE_COMPONENT=true
   -J URL_PORT	Retain the default port number. Default value: 443 Example: -J URL_PORT=443
   -J CUSTOM_CONNECTION_PROTOCOL	Retain the default connection protocol. Default value: https Example: -J CUSTOM_CONNECTION_PROTOCOL=https

9. Open a shell prompt and navigate to the directory that contains the installation files. Example: /BCO/Disk1

10. Launch the silent installer:

    ./setup.sh -i silent -DOPTIONS_FILE=<file path>/BCO_ADDITIONAL_SERVER_REMOTE_EE.txt


    If the path contains spaces, enclose the path and options file name in quotation marks.

    After the installation is complete a confirmation message about successful installation is displayed.

11. Verify the installation:

    1. Log in with the user credentials that you used for installation. Example: cpit.

    2. Navigate to the installation directory. Example: /opt/bmc/BCO.

    3. Run the command: ./cpit status

12. Verify that the Scheduler service is running.

Install BMC Helix Client Gateway to establish a connection between BMC Cloud and your on-premises datacenter. BMC Helix Client Gateway is a non-VPN solution to securely connect to your on-premises BMC Helix services. You must install and configure a small client in your environment to facilitate this connection. For information about downloading and installing BMC Helix Client Gateway, see BMC Helix Client Gateway connectivity.

Contact BMC Customer Support after installing BMC Helix Client Gateway. BMC will assist you in configuring BMC Helix Client Gateway and testing its connection with BMC Cloud.

Preparing to install the Gateway Server

• Install the Gateway Server on the same computer where you have installed the Remote ETL Engine.
• Use a non-privileged and non-root user account for installing the Gateway Server. The non-privileged user must be created on the installation computer before running the installer and must have access to use cron on the computer. This user owns all the Gateway Server files and runs most of the processes. The installer does not create the installation owner.
• KSH library must be available on the Linux system where you want to install the Gateway Server.

Installing the Gateway Server

1. Log in as a non-root user to the computer where you want to install the Gateway Server and create a temporary directory. Example: HOCO_temp/BCO/Disk1.
   
2. Download the installation files from ftp://<customerprefix>-opt-ftp.onbmc.com/FromOnDemand/optimize-onprem.
3. Extract the Gateway Server installation files to the temporary directory by running this command:
   tar -xvf TSCO_GatewayServer_<version>_Linux.tar
4. In the directory where you extracted the Gateway Server installation files, navigate to the ./BCO/Disk1/silent directory.

5. Copy the HO_ConsoleSilentInstallOptions.txt file to a new directory.
   This file contains the installation properties that are specified as name and value pairs with prefix tags. You can retain the default values or modify the properties of the file according to your preferences as listed below, and save the file. However, BMC recommends retaining the default values in the options file for the installation.

    Property details

   Prefix tags for the silent installation properties:

   • -P: Precedes the directory properties

   • -J: Precedes the Java properties

   Property name	Description
   Installation settings
   -P installLocation	Installation directory path. Default value: /opt/bmc/Patrol3 Example: -P installLocation=/opt/bmc/Patrol3
   -J BPA_USER	Username for installing the Gateway Server. Default value if not set: perform (when running the installation as root) current user (when running installation as any current non-root user) Example: -J BPA_USER=cpit
   Advanced settings
   -J PERFORM_AGENT_START_UNIX	Start the Capacity Agent after installing the Gateway Server. Default value: False Example: -J PERFORM_AGENT_START_UNIX=FALSE
   -J PERFORM_AGENT_ENABLE_HISTORY	Enable the collection of historical data. Default value: True Example: J PERFORM_AGENT_ENABLE_HISTORY=TRUE
   -J PERFORM_CREATE_DEFAULT_LINK	Create the best1_default link in the /usr/adm directory. Default value: True Example: -J PERFORM_CREATE_DEFAULT_LINK=TRUE
   -J PERFORM_AGENT_PORT	Port number to be used by the Capacity Agent. Default value: 6767 Example: -J PERFORM_AGENT_PORT=6767
   -J PERFORM_INVESTIGATE_PORT	Port number to be used by the Investigate. Default value: 6768 Example: -J PERFORM_INVESTIGATE_PORT=6768
   -J PERFORM_GENERAL_MANAGER_PORT	Port number to be used by the General Manager. Default value: 10129 Example: -J PERFORM_GENERAL_MANAGER_PORT=10129
   -J PERFORM_AGENT_SERVICE_PORT	Port number to be used by the Agent service. Default value: 10128 Example: -J PERFORM_AGENT_SERVICE_PORT=10128
   -J GWS_PROXY_PORT	Port number to be used by the Gateway Server proxy. Default value: 10130 Example: -J GWS_PROXY_PORT=10130
   -J PERFORM_AGENT_HISTORY_REPOSITORY	Directory path to store the historical data. Default value: /opt/bmc/Patrol3/perform/history Example: -J PERFORM_AGENT_HISTORY_REPOSITORY=/opt/bmc/Patrol3/perform/history
   -J PERFORM_AGENT_COLLECT_REPOSITORY	Directory path to store the data collected daily. Default value: /opt/bmc/Patrol3/perform/collect Example: -J PERFORM_AGENT_COLLECT_REPOSITORY=/opt/bmc/Patrol3/perform/collect
   -J PERFORM_AGENT_SD_LOCATION	Directory path for installing the service daemon. Default value: /etc/bgs/SD Example: -J PERFORM_AGENT_SD_LOCATION=/etc/bgs/SD
   -J PERFORM_PROCESS_LENGTH	Process command length that the Agent must collect. Default value: 2048 Example: -J PERFORM_PROCESS_LENGTH=2048
   -J PERFORM_SECURITY_CONFIGURATION_TYPE	Security level that you want to use for communication between the Gateway Server and the Helix Capacity Optimization Console. Use ADVANCED when a direct network connection does not exist between the Agent and the Gateway Server. Default value: BASIC Example: -J PERFORM_SECURITY_CONFIGURATION_TYPE=BASIC
   -J PERFORM_UNIX_MIGRATE_AGENT_LIST	Migrate the agent list to use the latest Gateway Server being installed. Default value: True Example: -J PERFORM_UNIX_MIGRATE_AGENT_LIST=TRUE
   -J PERFORM_UNIX_UNINSTALL	Uninstall the existing version of the Gateway Server if detected. Default value: False Example: -J PERFORM_UNIX_UNINSTALL=FALSE
   -J PERFORM_PATROL_CONFIGURATION_TYPE	Should the existing PATROL agent interact with the embedded Capacity Agent being installed. Default value: NONE, which indicates you do not want to change the existing PATROL agent configuration. Example: -J PERFORM_PATROL_CONFIGURATION_TYPE=NONE
   -J PATROL_AGENT_HOME	Path where the PATROL agent is installed. (Applicable only if you configure the PATROL agent to communicate with the new Capacity Agent.) Default value: /opt/bmc/Patrol Example: -J PATROL_AGENT_HOME=/opt/bmc/Patrol
   -J PERFORM_CONFIGURE_SERVICE_STANDALONE	Run the service daemon process separately or with the inetd or xinetd service. Default value: False Example: -J PERFORM_CONFIGURE_SERVICE_STANDALONE=FALSE
   -J PERFORM_AGENT_IP_TYPE	Preference for the protocol, whether IPv4 or IPv6. Default value: 46, which indicates that the IPv6 protocol must be used only when IPv4 protocol is not available. To use IPv6, type 64. Example: -J PERFORM_AGENT_IP_TYPE=46
   Do not modify these settings
   -J INSTALLATION_TYPE	Retain the default selection to install the Gateway Server using Custom installation Example: -J INSTALLATION_TYPE=Custom
   -J INSTALL_BPA_AGENT	Retain the default selection to install the Capacity Agent. Example: INSTALL_BPA_AGENT=true
   -J INSTALL_BPA_CONSOLE	Retain the default value as true for installing the Gateway Server. Example: -J INSTALL_BPA_CONSOLE=TRUE
   -J CONNECTION_TYPE	Retain the default value for the protocol to be used for communication. Default value: HTTP Example: -J CONNECTION_TYPE=HTTP
   -J BIND_TO_LOOPBACK	Retain the default value for binding the service to the loopback address. Default value: 1 Example: -J BIND_TO_LOOPBACK=1

6. Change to the directory that contains the extracted Gateway Server installation files.

7. Start the installation process by running the following command:
   ./setup.sh -i silent -DOPTIONS_FILE=<fileLocationPath>/HO_ConsoleSilentInstallOptions.txt
The variable <fileLocationPath> represents the location where you saved the options file. If the path contains spaces, enclose the path and options file name in quotation marks.

8. If the installation is executed as a non-root user, the /installLocation/b1configVVVV.sh script (where vvvv is the Gateway Server version) must be executed as the root user to complete the installation.  The b1configVVVV.sh script is executed automatically when the Gateway Server install is done by the root user.

9. Verify that the installation is successful.

    To verify that the GatewayManagerServer process is running

   1. Run this command to verify that port 10129 is enabled on the Gateway Server:
      netstat -an |grep 10129
   2. Run one of the following commands to verify that the General Manager service is running:
      1. For computers that have telnet:
         telnet <GW_server_hostname> 10129
      2. For computers that do not have telnet:
         export BEST1_HOME=/usr/adm/best1_V.V.VV (V.V.VV is the Gateway Server version)
         $BEST1_HOME/bgs/bin/GeneralManagerClient -s localhost:10129 getConsoleVersion
   3. Run this command to verify that the GeneralManagerServer process is up and running:
      ps -ef |grep GeneralManagerServer
    To verify data collection

   1. To configure the BEST1_HOME environment variable, run the following commands:
      

      1. Change to the non-root user.
      2. Run the following commands:
         
         • For the Korn or Bourne shell:
           
           1. BEST1_HOME=usr/adm/best1_<version>
           2. export BEST1_HOME
         • For the C shell:
           setenv BEST1_HOME usr/adm/best1_<version>
where version indicates the version of the installed product component.

   2. To test the connection to the Agent, run the following command:
      $BEST1_HOME/bgs/scripts/best1collect -n <targetHostName> -q
      The following example shows the result of running this command on the clm-pun-015565 system:
      

      
      

      [perform@clm-pun-032515 ~]$ $BEST1_HOME/bgs/scripts/best1collect -n clm-pun-015565 -q 
      best1collect on clm-pun-032515: requesting Update Status on clm-pun-032515 ... 
      Wed Mar 15 00:06:00 2017 
      Agent Query Request Starting(/usr/adm/best1_10.7.00/bgs/monitor/log/clm-pun-032515-bgsagent_6767.als) Collect Instance Node Started Started Name Name Name By On 
      ------------ -------------- ---------- ---------- ----------------- ------------ -------------- ---------- ---------- ----------------- 
      *Node: clm-pun-032515 has acknowledged Query request successfully.

      
      

   3. To initiate a data collection request, run the following command. Replace <nodeName> with the name of the system where you want to initiate the collection request.
      $BEST1_HOME/bgs/scripts/best1collect -n <nodeName> -e +15 -d /tmp -w 1
      

      The following example shows the result of running this command on the clm-pun-015565 system:

      [perform@clm-pun-032515 scripts]$ $BEST1_HOME/bgs/scripts/best1collect -n clm-pun-015565 -e +15 -d /tmp -w 1 
      best1collect on clm-pun-032515: requesting Start Collect on clm-pun-015565 ... 
      best1collect requesting bgscollect start on clm-pun-015565... 
      Tue Mar 14 22:30:04 2017 
      best1collect: Collect process with instance noInstance will be started on node clm-pun-015565. Specified data repository: /tmp. 
      *Node: clm-pun-015565 has acknowledged Start request successfully. 

      
      

   4. To verify the status of data collection, run the following command:
      

      $BEST1_HOME/bgs/scripts/best1collect -n <nodeName> -Q

      The following example shows the result of running this command on the clm-pun-015565.bmc.com system:
      

      [perform@clm-pun-032515 scripts]$ $BEST1_HOME/bgs/scripts/best1collect -n clm-pun-015565.bmc.com -Q 
      best1collect on clm-pun-032515: requesting Update Status on clm-pun-015565.bmc.com ... 
      Tue Mar 14 22:35:50 2017 
      Agent Query Request Starting(/usr/adm/best1_10.7.00/bgs/monitor/log/clm-pun-015565-bgsagent_6767.als) Collect Instance Node Started Started Data Length Spill SampleState Config Active Active Term Term LagTime Name Name Name By On Repository HH:MM (Min) (Sec) Groups Data No Data Data No Data (Min) 
      ------------ -------------- ---------- --------------------------- ----------------------------- ------ ----- ------------------------ ------ ------ ------- ---- ------- ------- SYSTEM noInstance clm-pun-015565 perform Mar-14-2017.22.30/tmp 0:15 15 10 UDR_WRITE_ACTIVE 46 19 2 0 25 0 
      ------------ -------------- ---------- --------------------------- ----------------------------- ------ ----- ------------------------ ------ ------ ------- ---- ------- -----
      *Node: clm-pun-015565.bmc.com has acknowledged Query request successfully. 

   5. Verify that the Active Data parameter shows a value greater than 0, which indicates that the data collector is functioning as expected.

If you have installed the Gateway Server to collect data, you must install the Capacity Agent on the infrastructure that you want to monitor.

The Capacity Agent collects data from the managed systems in your environment. After collecting data, the Agent transfers it to the Gateway Server. You can install the Capacity Agent using a Shell script or a Docker container. For instructions, see Installing the Capacity Agent.