Installing remote components to collect on-premises data

This section describes how to install the Remote ETL Engine and related components that are needed to collect data from your environment. 

Step 1: Prepare a Linux virtual machine

For information about system requirements and sizing guidelines, see Planning.

Step 2: Install the Remote ETL Engine

This topic describes how to install the Remote ETL Engine silently. A sample response file with default settings is included in the Remote ETL Engine install image. Modify the sample file instead of creating a new file. The sample file also includes instructions on how to run the installation silently.

  1. Log in as a root user to the computer where you want to install the Remote ETL Engine, and create a temporary directory. Example: HOCO_temp.

  2. Download the installation files from ftp://<customerprefix>-opt-ftp.onbmc.com/FromOnDemand/optimize-onprem.

  3. Extract the downloaded files to the temporary directory. Ensure that the temporary directory is different from the installation directory to avoid any file sharing conflict.

  4. Generate the API key from the Helix Capacity Optimization Console. You will need to specify this key while configuring the bearer authorization token in the options file. For details on generating the API key, see Generating an API key for programmatic access

  5. Open the credentials.key file in a text editor, and copy the authorization token.
    "AUTHORIZATION":"<BEARER TOKEN>"

  6. At the shell prompt, change to the directory where you extracted the downloaded files. Example: HOCO_temp/BCO/Disk1.

  7. Navigate to the directory where you extracted the downloaded files and locate the sample response file. Example: HOCO_temp/BCO/Disk1/silentInstallTemplates/BCO_ADDITIONAL_SERVER_REMOTE_EE.txt.

  8. Edit the BCO_ADDITIONAL_SERVER_REMOTE_EE.txt file in a text editor.

  9. In the -J AUTH_TOKEN property, paste the token that you have copied in Step 4. Modify other properties of the file according to your preferences as listed below, and save the file.

     Property details

    Prefix tags for the silent installation properties:

    • -P: Precedes the directory properties

    • -A: Precedes the application features

    • -J: Precedes the Java properties

    Property nameDescription
    Installation settings

    -P installLocation

    Installation directory path.

    Default value: /opt/bmc/BCO

    Example: -P installLocation=/opt/bmc/BCO

    For root user

    -J BCO_USER

    Name of the user (new or existing).

    Default value: cpit

    Example: -J BCO_USER=cpit

    Helix host details
    -J URL_HOSTExternal data API server name. Example: -J URL_HOST=companyname-optapi.onbmc.com
    Authorization Bearer Token
    -J AUTH_TOKEN

    Bearer authorization key. This key is used to authenticate the user when connecting to the Helix Capacity Optimization Console. For instructions about generating the key, see Step 3.

    Example: -J AUTH_TOKEN=Bearer eyJ0eXAiOiJKV1QiLCJhbGciOiJSUzI1NiJ9.eyJzdWIiOiJBRE1JTiIsInJvbG...VfaWQiOjAsInJvbGVMe5yckarMjPg

    Proxy server connection
    -J IS_PROXY_CONFIGUREDConfigure the proxy server. Default value: false Example: -J IS_PROXY_CONFIGURED=false
    -J REE_PROXY_HOSTName of the proxy server host.

    -J REE_PROXY_HOST=<proxy_server_hostname>

    Example: -J REE_PROXY_HOST=vw-abc-dev01.bmc.com

    -J REE_PROXY_PORT

    Port number to communicate with the proxy server.

    Example: -J REE_PROXY_PORT=8090

    -J REE_PROXY_USER

    Proxy server user name.

    Example: -J REE_PROXY_USER=proxy_user

    -J REE_PROXY_SECPWD

    Proxy server password.

    Example: -J REE_PROXY_SECPWD=proxy_user_password

    Advanced properties
    -J IS_RECONFIGURATION_FLOW -J IS_REPAIR_FLOW

    To run the installer on the existing installation, choose one of the following modes. You can set only one of these properties to true as the installer can run only in one of these modes.

    • Reconfigure: Use the reconfigure mode to modify certain values that you had provided during installation, such as database connection or repository option, in the installed product. 
      -J IS_RECONFIGURATION_FLOW
      Default value: false
    • Re-installation: Use the re-installation mode to re-install the Application Server or ETL Engine. BMC recommends that you back up your database before starting the re-installation.
      -J IS_REPAIR_FLOW
      Default value: false
    Do not modify these settings

    -J INSTALLATION_TYPE_SELECTION

    Retain the default installation type.

    Default value: EXTERNAL_SERVER

    Example: -J INSTALLATION_TYPE_SELECTION=EXTERNAL_SERVER

    -A <feature name>

    Install the Remote ETL Engine component.

    Default value: featureCoreETL

    Example: -A featureCoreETL

    -J IS_REMOTE_COMPONENT

    Retain the default value.

    Default value: true

    Example: -J IS_REMOTE_COMPONENT=true

    -J URL_PORTRetain the default port number. Default value: 443 Example: -J URL_PORT=443
    -J CUSTOM_CONNECTION_PROTOCOLRetain the default connection protocol. Default value: https Example: -J CUSTOM_CONNECTION_PROTOCOL=https

  10. Open a shell prompt and navigate to the directory that contains the installation files. Example: /BCO/Disk1
  11. Launch the silent installer:

    ./setup.sh -i silent -DOPTIONS_FILE=<file path>/BCO_ADDITIONAL_SERVER_REMOTE_EE.txt

    If the path contains spaces, enclose the path and options file name in quotation marks.

    After the installation is complete a confirmation message about successful installation is displayed.

  12. Verify the installation:

    1. Log in with the user credentials that you used for installation. Example: cpit.

    2. Navigate to the installation directory. Example: /opt/bmc/BCO.

    3. Run the command: ./cpit status
  13. Verify that the Scheduler service is running.

The following steps (3, 4, and 5) are required only if you plan to use the Capacity Agent for collecting data.

Step 3: Install the BMC Helix Client Gateway

Install the BMC Helix Client Gateway to establish a connection between BMC Cloud and your on-premises datacenter. BMC Helix Client Gateway is a non-VPN solution to securely connect to your on-premises BMC Helix services. You must install a small client in your environment to facilitate this connection. For more information, see BMC Helix Client Gateway connectivity .

To use the BMC Helix Client Gateway, open a technical support issue, and BMC will provide the client installer. Depending on your requirements, BMC will help you plan for a single or multiple virtual machines.

Step 4: Install the Gateway Server

Preparing to install the Gateway Server

  • Install the Gateway Server on the same computer where you have installed the Remote ETL Engine.
  • Use a non-privileged and non-root user account for installing the Gateway Server. The non-privileged user must be created on the installation computer before running the installer and must have access to use cron on the computer. This user owns all the Gateway Server files and runs most of the processes. The installer does not create the installation owner.
  • KSH library must be available on the Linux system where you want to install the Gateway Server.

Installing the Gateway Server

  1. Log in as a non-root user to the computer where you want to install the Gateway Server and create a temporary directory. Example: HOCO_temp/BCO/Disk1.
  2. Download the installation files from ftp://<customerprefix>-opt-ftp.onbmc.com/FromOnDemand/optimize-onprem.
  3. Extract the Gateway Server installation files to the temporary directory by running this command:
    tar -xvf TSCO_GatewayServer_<version>_Linux.tar
  4. In the directory where you extracted the Gateway Server installation files, navigate to the ./BCO/Disk1/silent directory.
  5. Copy the HO_ConsoleSilentInstallOptions.txt file to a new directory.
    This file contains the installation properties that are specified as name and value pairs with prefix tags. You can retain the default values or modify the properties of the file according to your preferences as listed below, and save the file. However, BMC recommends retaining the default values in the options file for the installation.

     Property details

    Prefix tags for the silent installation properties:

    • -P: Precedes the directory properties

    • -J: Precedes the Java properties

    Property nameDescription
    Installation settings
    -P installLocation

    Installation directory path.

    Default value: /opt/bmc/Patrol3

    Example: -P installLocation=/opt/bmc/Patrol3

    -J BPA_USERUsername for installing the Gateway Server.

    Default value if not set:

    perform (when running the installation as root)

    current user (when running installation as any current non-root user)

    Example: -J BPA_USER=cpit

    Advanced settings
    -J PERFORM_AGENT_START_UNIXStart the Capacity Agent after installing the Gateway Server.
    Default value: True
    Example: -J PERFORM_AGENT_START_UNIX=TRUE
    -J PERFORM_AGENT_ENABLE_HISTORYEnable the collection of historical data.
    Default value: True
    Example: J PERFORM_AGENT_ENABLE_HISTORY=TRUE
    -J PERFORM_CREATE_DEFAULT_LINKCreate the best1_default link in the /usr/adm directory.
    Default value: True
    Example: -J PERFORM_CREATE_DEFAULT_LINK=TRUE
    -J PERFORM_AGENT_PORTPort number to be used by the Capacity Agent.
    Default value: 6767
    Example: -J PERFORM_AGENT_PORT=6767
    -J PERFORM_INVESTIGATE_PORTPort number to be used by the Investigate.
    Default value: 6768
    Example: -J PERFORM_INVESTIGATE_PORT=6768
    -J PERFORM_GENERAL_MANAGER_PORTPort number to be used by the General Manager.
    Default value: 10129
    Example: -J PERFORM_GENERAL_MANAGER_PORT=10129
    -J PERFORM_AGENT_SERVICE_PORTPort number to be used by the Agent service.
    Default value: 10128
    Example: -J PERFORM_AGENT_SERVICE_PORT=10128
    -J GWS_PROXY_PORTPort number to be used by the Gateway Server proxy.
    Default value: 10130
    Example: -J GWS_PROXY_PORT=10130
    -J PERFORM_AGENT_HISTORY_REPOSITORYDirectory path to store the historical data.
    Default value: /opt/bmc/Patrol3/perform/history
    Example: -J PERFORM_AGENT_HISTORY_REPOSITORY=/opt/bmc/Patrol3/perform/history
    -J PERFORM_AGENT_COLLECT_REPOSITORYDirectory path to store the data collected daily.
    Default value: /opt/bmc/Patrol3/perform/collect
    Example: -J PERFORM_AGENT_COLLECT_REPOSITORY=/opt/bmc/Patrol3/perform/collect
    -J PERFORM_AGENT_SD_LOCATIONDirectory path for installing the service daemon. Default value: /etc/bgs/SD Example: -J PERFORM_AGENT_SD_LOCATION=/etc/bgs/SD
    -J PERFORM_PROCESS_LENGTHProcess command length that the Agent must collect.
    Default value: 2048
    Example: -J PERFORM_PROCESS_LENGTH=2048
    -J PERFORM_SECURITY_CONFIGURATION_TYPE

    Security level that you want to use for communication between the Gateway Server and the Helix Capacity Optimization Console. Use ADVANCED when a direct network connection does not exist between the Agent and the Gateway Server.
    Default value: BASIC
    Example: -J PERFORM_SECURITY_CONFIGURATION_TYPE=BASIC

    -J PERFORM_UNIX_MIGRATE_AGENT_LISTMigrate the agent list to use the latest Gateway Server being installed.
    Default value: True
    Example: -J PERFORM_UNIX_MIGRATE_AGENT_LIST=TRUE
    -J PERFORM_UNIX_UNINSTALLUninstall the existing version of the Gateway Server if detected.
    Default value: False
    Example: -J PERFORM_UNIX_UNINSTALL=FALSE
    -J PERFORM_PATROL_CONFIGURATION_TYPEShould the existing PATROL agent interact with the embedded Capacity Agent being installed.
    Default value: NONE, which indicates you do not want to change the existing PATROL agent configuration.
    Example: -J PERFORM_PATROL_CONFIGURATION_TYPE=NONE
    -J PATROL_AGENT_HOMEPath where the PATROL agent is installed. (Applicable only if you configure the PATROL agent to communicate with the new Capacity Agent.)
    Default value: /opt/bmc/Patrol
    Example: -J PATROL_AGENT_HOME=/opt/bmc/Patrol
    -J PERFORM_CONFIGURE_SERVICE_STANDALONERun the service daemon process separately or with the inetd or xinetd service.
    Default value: False
    Example: -J PERFORM_CONFIGURE_SERVICE_STANDALONE=FALSE
    -J PERFORM_AGENT_IP_TYPEPreference for the protocol, whether IPv4 or IPv6.
    Default value: 46, which indicates that the IPv6 protocol must be used only when IPv4 protocol is not available. To use IPv6, type 64.
    Example: -J PERFORM_AGENT_IP_TYPE=46
    Do not modify these settings
    -J INSTALLATION_TYPERetain the default selection to install the Gateway Server using Custom installation Example: -J INSTALLATION_TYPE=Custom
    -J INSTALL_BPA_AGENTRetain the default selection to install the Capacity Agent.
    Example: INSTALL_BPA_AGENT=true
    -J INSTALL_BPA_CONSOLERetain the default value as true for installing the Gateway Server. Example: -J INSTALL_BPA_CONSOLE=TRUE
    -J CONNECTION_TYPERetain the default value for the protocol to be used for communication. Default value: HTTP Example: -J CONNECTION_TYPE=HTTP
    -J BIND_TO_LOOPBACK

    Retain the default value for binding the service to the loopback address. Default value: 1

    Example: -J BIND_TO_LOOPBACK=1

  6. Change to the directory that contains the extracted Gateway Server installation files.

  7. Start the installation process by running the following command:
    ./setup.sh -i silent -DOPTIONS_FILE=<fileLocationPath>/HO_ConsoleSilentInstallOptions.txt
    The variable <fileLocationPath> represents the location where you saved the options file. If the path contains spaces, enclose the path and options file name in quotation marks.

  8. If the installation is executed as a non-root user, the /installLocation/b1configVVVV.sh script (where vvvv is the Gateway Server version) must be executed as the root user to complete the installation.  The b1configVVVV.sh script is executed automatically when the Gateway Server install is done by the root user.
  9. Verify that the installation is successful.

     To verify that the GatewayManagerServer process is running
    1. Run this command to verify that port 10129 is enabled on the Gateway Server:
      netstat -an |grep 10129
    2. Run one of the following commands to verify that the General Manager service is running:
      1. For computers that have telnet:
        telnet <GW_server_hostname> 10129
      2. For computers that do not have telnet:
        export BEST1_HOME=/usr/adm/best1_V.V.VV (V.V.VV is the Gateway Server version)
        $BEST1_HOME/bgs/bin/GeneralManagerClient -s localhost:10129 getConsoleVersion
    3. Run this command to verify that the GeneralManagerServer process is up and running:
      ps -ef |grep GeneralManagerServer
     To verify data collection
    1. To configure the BEST1_HOME environment variable, run the following commands:

      1. Change to the non-root user.
      2. Run the following commands:
        • For the Korn or Bourne shell:
          1. BEST1_HOME=usr/adm/best1_<version>
          2. export BEST1_HOME
        • For the C shell:
          setenv BEST1_HOME usr/adm/best1_<version>
          where version indicates the version of the installed product component.
    2. To test the connection to the Agent, run the following command:
      $BEST1_HOME/bgs/scripts/best1collect -n <targetHostName> -q
      The following example shows the result of running this command on the clm-pun-015565 system:


      [perform@clm-pun-032515 ~]$ $BEST1_HOME/bgs/scripts/best1collect -n clm-pun-015565 -q 
      best1collect on clm-pun-032515: requesting Update Status on clm-pun-032515 ... 
      Wed Mar 15 00:06:00 2017 
      Agent Query Request Starting(/usr/adm/best1_10.7.00/bgs/monitor/log/clm-pun-032515-bgsagent_6767.als) Collect Instance Node Started Started Name Name Name By On 
      ------------ -------------- ---------- ---------- ----------------- ------------ -------------- ---------- ---------- ----------------- 
      *Node: clm-pun-032515 has acknowledged Query request successfully.


    3. To initiate a data collection request, run the following command. Replace <nodeName> with the name of the system where you want to initiate the collection request.
      $BEST1_HOME/bgs/scripts/best1collect -n <nodeName> -e +15 -d /tmp -w 1

      The following example shows the result of running this command on the clm-pun-015565 system:

      [perform@clm-pun-032515 scripts]$ $BEST1_HOME/bgs/scripts/best1collect -n clm-pun-015565 -e +15 -d /tmp -w 1 
      best1collect on clm-pun-032515: requesting Start Collect on clm-pun-015565 ... 
      best1collect requesting bgscollect start on clm-pun-015565... 
      Tue Mar 14 22:30:04 2017 
      best1collect: Collect process with instance noInstance will be started on node clm-pun-015565. Specified data repository: /tmp. 
      *Node: clm-pun-015565 has acknowledged Start request successfully. 


    4. To verify the status of data collection, run the following command:

      $BEST1_HOME/bgs/scripts/best1collect -n <nodeName> -Q

      The following example shows the result of running this command on the clm-pun-015565.bmc.com system:

      [perform@clm-pun-032515 scripts]$ $BEST1_HOME/bgs/scripts/best1collect -n clm-pun-015565.bmc.com -Q 
      best1collect on clm-pun-032515: requesting Update Status on clm-pun-015565.bmc.com ... 
      Tue Mar 14 22:35:50 2017 
      Agent Query Request Starting(/usr/adm/best1_10.7.00/bgs/monitor/log/clm-pun-015565-bgsagent_6767.als) Collect Instance Node Started Started Data Length Spill SampleState Config Active Active Term Term LagTime Name Name Name By On Repository HH:MM (Min) (Sec) Groups Data No Data Data No Data (Min) 
      ------------ -------------- ---------- --------------------------- ----------------------------- ------ ----- ------------------------ ------ ------ ------- ---- ------- ------- SYSTEM noInstance clm-pun-015565 perform Mar-14-2017.22.30/tmp 0:15 15 10 UDR_WRITE_ACTIVE 46 19 2 0 25 0 
      ------------ -------------- ---------- --------------------------- ----------------------------- ------ ----- ------------------------ ------ ------ ------- ---- ------- -----
      *Node: clm-pun-015565.bmc.com has acknowledged Query request successfully. 
    5. Verify that the Active Data parameter shows a value greater than 0, which indicates that the data collector is functioning as expected.


Step 5: Install the Capacity Agent

If you have installed the Gateway Server to collect data, you must install the Capacity Agent on the infrastructure that you want to monitor.

The Capacity Agent collects data from the managed systems in your environment. After collecting data, the Agent transfers it to the Gateway Server. You can install the Capacity Agent using a Shell script or a Docker container. For instructions, see Installing the Capacity Agent.

Was this page helpful? Yes No Submitting... Thank you

Comments

  1. Brandon Dias

    Where are the details related to upgrade of remote components - Gateway Server and Remote ETL Engine Server?

    Jun 24, 2020 07:13
  2. Brandon Dias

    The instructions for the REE install above describe how to install the REE using the silent install method, where are the instructions if you dont want to install it silently? Or is that the only option?

    Jun 24, 2020 08:49
  3. Brandon Dias

    The Dashboard page: https://docs.bmc.com/docs/helixcapacityoptimization/hco2002/accessing-and-navigating-the-helix-capacity-optimization-dashboard-914170333.html?src=search shows a screenshot of the Dashboard, with a link to install the ETL server on-prem. Where are the docs which describes this process, install of the on-prem ETL server, without silent install option?

    Jun 24, 2020 08:52