Moving or duplicating a TrueSight Server Automation environment

This topic outlines the procedure for moving or duplicating a TrueSight Server Automation environment. This includes the TrueSight Server Automation application, file, and PXE/TFTP servers.  

You might want to move or replicate a TrueSight Server Automation environment for the following reasons:

  • To create a test environment to prepare for an upgrade, training or other testing
  • To move TrueSight Server Automation infrastructure to new hardware.

When performing the upgrade of TrueSight Server Automation, it is usually advantageous to attempt the upgrade in a copy of the production environment — this will give you an idea of how long the actual upgrade will take and identify any possible errors that can be resolved by BMC Support in a less time-critical environment.  Additionally, the 'copy and upgrade' method can be used for the actual upgrade or for creating a new environment.  The steps below are meant to cover the high-level steps that you must perform, and the specifics will likely vary, depending on the database in use, the operating systems involved, and the particular environment details.

Some permutation of the below steps can be used to cover a few different use cases:

  • Testing the upgrade in a tear-down environment before running in the live production environment
  • Creating a clone of the existing environment on new hardware before decommissioning the old environment
  • Moving the old environment to new hardware and upgrading in one swoop

The following sections discuss the necessary tasks:

Before You Begin

Before moving or replicating your TrueSight Server Automation environment, note the following issues:

  • You will maybe need to temporarily stop your TrueSight Server Automation Application Server to create a backup of your database and file server, depending on the backup mechanisms being used. This will ensure that both backups are synchronized to the same date and time.
    • Therefore, you will need to notify any TrueSight Server Automation users that the system will be down during this process. The amount of time that the application server will be down depends on how quickly you are able to copy your file server and database. This time period can vary, depending on how much data is contained in each of these infrastructure pieces.
  • Decide what you are going to move or replicate. In some cases, you might want to move only the file server. In other cases you might want to replicate your entire infrastructure. The following sections assume that you are moving or replicating your entire environment, so if a particular piece does not apply to what you are moving or replicating, skip that step.

Important

Do not clone the operating system when you move or replicate the Application Server to a different host.

Duplicating the installation infrastructure

Perform the following steps in the listed order. For example, make sure that the new TrueSight Server Automation database instance/schema is available before installing the new application server. These steps apply to a physical or virtual environment.

Prerequisites

Ensure that your environment includes the following elements:

  • A server in which to install the TrueSight Server Automation Application Server
  • A server in which to install the RSCD Agent for the File Server (which may also be the same server as the Application Server)
  • Enough space on the File Server to match the space for the current Production File Server
  • A database of the same type and version as the current Production database in which to import the exported database

Moving the Database

To move the database, perform the following steps:

  1. In the new database server, create the TrueSight Server Automation database or tablespaces and the TrueSight Server Automation database user.
  2. Export the database/schema from the old TrueSight Server Automation Database.  There are many utilities that can do this, and several examples are noted below.
    • Oracle: Use the Oracle 'data pump' utility, as in the following example.
      expdp [sys username]/[password]@[sid] schemas=[bladelogic schema] directory=[test_dir] dumpfile=[bladelogic schema].dmpdp logfile=expdp_[bladelogic schema]_[date].log flashback_time="\"to_timestamp(to_char(sysdate,'mm-dd-yyyy hh24:mi:ss'), 'mm-dd-yyyy hh24:mi:ss')\"" 
    • MSSQL
      1. Shut down all the Application Servers.
      2. Set the database to single-user mode. You can choose one of the following methods to do this.
        • Through the SQL Management Studio:
          1. In Object Explorer, connect to an instance of the SQL Server Database Engine, and then expand that instance.
          2. Right-click the database to change, and then click Properties.
          3. In the Database Properties dialog box, click the Options page.
          4. From the Restrict Access option, select Single.
          5. If other users are connected to the database, an Open Connections message appears. To change the property and close all other connections, click Yes.

        • By running a SQL query:

          USE master;
          GO
          ALTER DATABASE BLADELOGIC
          SET SINGLE_USER
          WITH ROLLBACK BLADELOGIC;
          GO
          ALTER DATABASE BLADELOGIC
          SET READ_ONLY;
          GO
          ALTER DATABASE BLADELOGIC
          SET MULTI_USER;
          GO
      3. Right click the database and select Tasks and then Back Up, and perform a full backup.

  3. Import the core database into the new database server w/ the corresponding import tool.  An example is shown below.

    Note

    If the target test environment already has working Application Servers and you want to preserve their configurations, then prior to the database import, export the contents of the application_server_deployments and service_config_property_value tables from the test database. These tables store the configuration data for the application servers.

    • Oracle: Use the Oracle 'data pump' utility, as in the following example.
      impdp [sys username]/[password]@[sid] dumpfile=[bladelogic schema].dmpdp logfile=impdp_[bladelogic schema]_[date].log transform=oid:n full=n schemas=[old schema name] directory=[test_dir] table_exists_action=replace remap_tablespace=[old bladelogic tablespace:new bladelogic tablespace],[old bladelogic index tablespace:new bladelogic index tablespace]
      remap_schema=[old bladelogic schema:new bladelogic schema]  
    • MSSQL: Import or restore the backup that was taken from the production database.

Moving the File Server

To move the file server, perform the following steps:

  1. Install an RSCD agent on the server that will function as a File Server.

    Note

    If you are using a non-Windows environment, you can skip this step, as the RSCD Agent is included in the Application Server installation that will be performed in the next section.

  2. Copy the file server storage location from the old location to the new location.  This can be done with a variety of utilities such as dsync, rsync and NAS or SAN replication tools.  The file server replication should be paired to the database replication.
  3. If the application server is not being moved then follow the Changing-the-location-of-the-file-server to update the application server configuration and database with the location of the new file server, otherwise wait to perform this step when the new application server is online.

Using the Application Server(s) in the new environment

To use the application server(s) in the new environment with the imported database, perform the following steps:

  1. If the target environment already has one or more TrueSight Server Automation Application Servers, perform one of the following steps (a or b):
    1. Import the test application server configurations that were exported in step 3 of Moving the database above.
      1. After the database import, delete the contents of the application_server_deployments and service_config_property_value tables.
      2. Delete any deployment directories in NSH/br/deployments except _launcher and _template.
      3. Import or insert the table export data from the application_server_deployments and service_config_property_value tables that you exported just prior to the database import.
    2. Create new deployments in the newly imported database.
      1. Delete any deployment directories in NSH/br/deployments except _launcher and _template on the test environment application servers.
      2. Run a single blasadmin command against template on each application server to trigger the registration of the application server with the newly imported database
        blasadmin -s _template set Database DriverClassoracle.jdbc.driver.OracleDriver (or the appropriate driver class)
        For more information about using the blasadmin commands, see Using blasadmin commands.
      3. Create the desired instances with the blasadmin create command.

  2. If this is not an upgrade related to a Disaster Recovery site, soft delete the old Application Servers from the imported database.

    Important: Adding the relevant backup date and time

    When you run the following SQL statements, modify the 'bkup08232019_' to the relevant backup date and time.

    (Oracle only)

    update application_server set is_deleted = 1;
    COMMIT;
    update application_server_model set is_deleted = 1;
    COMMIT;

    UPDATE APPLICATION_SERVER t1
    SET APP_SERVER_NAME = 'bkup08232019_' || (SELECT APP_SERVER_NAME
    FROM APPLICATION_SERVER t2
    WHERE t1.APPLICATION_SERVER_ID = t2.APPLICATION_SERVER_ID
    AND t2.IS_DELETED = 1)
    WHERE EXISTS (SELECT 1
    FROM APPLICATION_SERVER t2
    WHERE t1.APPLICATION_SERVER_ID = t2.APPLICATION_SERVER_ID
    AND t2.IS_DELETED = 1
    );
    COMMIT;


    UPDATE APPLICATION_SERVER_MODEL t1
    SET NAME = 'bkup08232019_' || (SELECT NAME
    FROM APPLICATION_SERVER_MODEL t2
    WHERE t1.APPLICATION_SERVER_MODEL_ID = t2.APPLICATION_SERVER_MODEL_ID
    AND t2.IS_DELETED = 1)
    WHERE EXISTS (
    SELECT 1
    FROM APPLICATION_SERVER_MODEL t2
    WHERE t1.APPLICATION_SERVER_MODEL_ID = t2.APPLICATION_SERVER_MODEL_ID
    AND t2.IS_DELETED = 1);
    COMMIT;

    (SQL Server only)

    update application_server set is_deleted = 1;
    GO
    update application_server_model set is_deleted = 1;
    GO
    UPDATE APPLICATION_SERVER
    SET APP_SERVER_NAME = 'bkup08232019_' + (select t2.APP_SERVER_NAME
    from APPLICATION_SERVER t2
    where t2.APPLICATION_SERVER_ID = t1.APPLICATION_SERVER_ID
    and t2.IS_DELETED = 1
    )
    FROM APPLICATION_SERVER t1;
    GO
    UPDATE t1
    SET t1.NAME = 'bkup08232019_' + (select t2.NAME
    from APPLICATION_SERVER_MODEL t2
    where t2.APPLICATION_SERVER_MODEL_ID = t1.APPLICATION_SERVER_MODEL_ID
    and t2.IS_DELETED = 1
    )
    FROM APPLICATION_SERVER_MODEL t1;
    GO

    Note

    In the code example, the substring bkup08232019_ must reflect the current date and be unique. You might receive an error message if same sql query is run twice on the same database.

  3. If the target environment has Application Servers of a version earlier than 8.6, and the database is already version 8.6 or later, run the configurator utility before upgrading, in order to persist configuration data to the database. This is necessary only when upgrading a Disaster Recovery environment from a release earlier than 8.6 to 8.6 or later and the Disaster Recovery Application Servers are registered in the application_server and application_server_model tables.
  4. If the target environment does not have any Application servers and the database is already present, perform the following steps
    1. Install the Application Server software on the new Application Server(s) using the individual component installers, not the unified product installer. 
    2. Using the blasadmin utility, configure the Database section parameters noted below.  These settings will be for the new database server.blasadmin -a set database ConnectionString <ConnectionString>blasadmin -a set database DriverClass <DriverClass>blasadmin -a set database UserId <user_name>blasadmin -a set database Password <Password>For more information about using the blasadmin commands, see Using blasadmin commands.
  5. Copy the files listed in the Files section of the Backup and Restore of the TrueSight Server Automation Environment document from the old application server to the new application server(s) except the bladelogic.keystore file.  Leave the bladelogic.keystore that we created during install on the new application server.
  6. Follow the Changing-the-location-of-the-file-server to update the application server configuration and database with the location of the new file server.

Provisioning

To move the provisioning environment, perform the following steps:

  1. If replicating your provisioning environment, install the TrueSight Server Automation PXE and TFTP server.
  2. Copy your provisioning data store to the new server.
  3. Replicate any custom application server settings by running <bl_install_dir>/br/blasadmin.
  4. Update the datastore locations to point to the new locations in the new environment.


Post-move configurations


After your environment has been moved, there are some final steps that you will want to make to ensure that there are no conflicts with your existing environment.

Updating Extended Object Locations

If there are Extended Objects that reference the old file server or other location these must be updated as well using the update_extended_object_script_location included in the external-files.zip for the release.

Updating Remote Depot Object Locations

If there are Depot objects whose payload is stored on a remote NSH path and not the file server, and those payloads are also moved in the new environment, the depot_object_location.remote_path column will need to be updated as well.

Preventing access from the new environment to the existing Application Servers

There are a few ways to accomplish this. One way to handle this is to generate a self-signed certificate for an Application Server in the new environment. Another way is to restrict network access between the two environments; this will prevent accidentally connecting to the original environment.

Removing Job Schedules

If the new environment is being used for testing you will want to disable scheduled jobs so they will not run against the registered servers.  This can be done with a simple query:

(Oracle only)

update schedule set is_deleted = 1;
update job_run set progress_status_id = 7 where progress_status_id in(1, 2, 3, 5, 6);
commit; 

(SQL Server only)

update schedule set is_deleted=1;
GO
update job_run set progress_status_id = 7 where progress_status_id in(1, 2, 3, 5, 6);
GO

Setting the is_deleted flag is a non-recoverable operation, the schedules will be permanently disabled in the new environment. This is typically desirable for testing environments.

Update Catalog Locations (Optional)

If you need to make use of the patch catalogs in the testing environment then you can also update the catalog location settings with a database query:

(Oracle only)

update job_option_value set option_value = replace (option_value, 'OLDNAME','NEWNAME') where option_value like '%OLDNAME%';

commit;

(SQL Server only)

update job_option_value set option_value = replace (option_value,@OLDNAME,@NEWNAME) where option_value like '%' + OLDNAME + '%'

Where OLDNAME is the old path to the catalog and NEWNAME is the path to the new location of the catalog.

Removing Notifications

If the new environment is being used for testing you will want to disable any job notifications to avoid confusing when testing jobs in the new environment. This can be done with a simple query:

update notification set email_to='test123@example.com';

commit; (Oracle Only)

Removing Routing Rules

If the new environment is being used for testing or the existing repeaters, appservers and SOCKs proxies will no longer be needed, then the routing rules can be removed with a few queries:

Delete SOCKS Proxy assigned in Network routing rule
update routing_policy set is_deleted = 1 where routing_policy_type_id = 1;

Delete Job Server assigned in job routing rule.

update routing_policy set is_deleted = 1 where routing_policy_type_id = 2

Delete Repeater assigned in Repeater routing rule.
update routing_policy set is_deleted =1 where routing_policy_type_id = 3;

commit; (Oracle only)

Preventing access from the new environment to existing managed servers

If the new environment is being used for testing you may want to prevent the new application servers from talking to your existing targets.  This is possible by pushing out an exports file from the original appserver/environment to all target servers that allows connections from only the original application servers and any NSH client systems you need to allow connections from.

Decommissioning Servers (optional)

  1. Decommission any servers that are in the development environment that you will not manage. This will prevent any confusion for anyone that might log into the duplicated environment.

    Note

    If you have an 8.x environment AND have the Licensing Portal configured, make sure that you configure the Licensing Portal to not re-claim any licenses when you decommission servers. If you do, you will need to re-license your servers in the original environment that you copied from.

  2. Ensure that only the servers that are expected to be registered in the duplicated environment remain.


Final Steps

After completing the steps in the new environment, start up the Application Server services and log on to the TrueSight Server Automation Console.  Ensure that you start up all Application Servers in the duplicate environment before performing any upgrade or other work. This ensures that the duplicate Application Servers are registered in the duplicate database, and that they can communicate with the duplicate file server and with the other Application Servers in the duplicate environment.

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*