Backing up and restoring a cluster

The appliance backup feature enables you to back up an appliance. You can restore the backed-up data when required. The backup can be made to the local file system, or to a remote destination over ssh or via a Windows share, so little local disk space is required. You can also choose to encrypt the backup. Security data is not included in a backup that is not encrypted. You can choose to be notified by email, if configured, of the completion and result (success or failure) of a backup task.

If you use appliance backup on a cluster, the entire cluster is backed up. If you choose to backup to local filesystem (the On Member option), each cluster member is backed up onto its local filesystem. When you backup a cluster to a remote destination, the appliance backup feature creates a single backup in which each machine's backup is contained in a subdirectory.

You can cancel an in-progress cluster backup, but only from original session on the UI of the machine from which the backup was started.

Important notices

Read and take note of the following notices before using the backup feature.

VMware snapshots and clusters

You can use the VMware snapshot tools to create a snapshot of all of the machines in a cluster. However, all machines must be shut down before starting the snapshot and must remain shut down until all of the snapshots are complete. This enables any internal cluster communication to be completed before the snapshot is created.

Destination system time must not be earlier than source when restoring a backup 

You must ensure that the current system time on the destination appliance is no earlier than that of the appliance on which the backup was created. If the modification times on the files contained in the backup are later than the system time when they are restored, the backup will hang. To recover at this point you must kill the backup process, correct the time, run tw_restore --fix-interrupted then repeat the restore using the tw_restore command line utility.

Backing up and CMDB synchronization

The backup contains the CMDB synchronization configuration. When a host has significantly changed so that its key has also changed, problems can be caused if a backup is restored before the changed host is rediscovered. In this case, on the next CMDB synchronization, duplicate hosts will be created in the CMDB representing the changed host, and the CIs representing the original hosts will never be deleted. To ensure that no duplicate hosts are created, you can delete and then recreate the BMC.ADDM dataset.

The backup contains the LDAP configuration. If the destination appliance cannot access the LDAP server, you must ensure that a local (non-LDAP) user belonging to the system and public groups is activated and successfully tested on the source appliance before making a backup.

If you choose to exclude the credential vault when backing up an appliance in which CMDB synchronization has been configured, the CMDB Sync page on the restored appliance displays the "This appliance has not been set up for synchronization with the Atrium CMDB" message. Once the Setup form is complete, filter and blackout window settings are restored.

Local backups 

We do not recommend using the On Members (local) backup option. The On Members backup type does not properly identify which member should receive which backup, and could result in service start failures after restore. You should use an SSH or SMB backup instead. See also the workaround (described below) that might be required for Windows share (SMB) backups.

Windows share (SMB) backups 

An OS level defect can prevent Windows share backups from completing. The verification step fails with the message "ERROR: Cannot read file ... Unexpected end of stream".

To workaround this problem:

1. Mount the share (that you want to backup to) on one of the cluster members, or on a spare Linux host.
2. Use SSH (or local) backup to the directory used to mount the Windows share.

Do not disable the verification step for Windows share backups. If you do so, and the backup fails because of this defect, you will receive no notification of the failure.

Backing up and restoring a cluster

The appliance backup feature replaces the appliance snapshot that was available in previous releases. On upgraded appliances only, if snapshots are still held in the filesystem, a banner and Remove Snapshots button is provided so you can remove the snapshots and release disk space.

During the backup, the services on all members of the cluster are shutdown for the backup and restarted when the backup is complete.

To create a backup of the cluster

1. From the main menu, click the Administration icon.  
   The Administration page displays.
2. In the Appliance section, click Backup & Restore.
   The Appliance Backup page opens. This has a panel in which you can configure the backup destination. It also shows details of the cluster size, the size of the backup for this member, and details of the contents of the backup.
   

3. Enter the details for the backup destination.

   The fields that can be completed are displayed or hidden depending on the backup type selection. Required fields are indicated with a red asterisk.

   Field Name	Details
   Backup Type	Select the destination type from the drop-down list. This can be one of the following: • On Members—The backup for each machine in the cluster is written to the $TIDEWAY/var/backup directory on that machine. Only one local (On Members) backup can be stored. We do not recommend using the On Members (local) backup option, see Local backups for more information.  • SSH—The backup is written to a remote server over ssh. • Windows Share—The backup is written to a Windows share.
   Notes	A free text area in which you can write notes about the backup.
   Host	The hostname or IP address of the remote server onto which to write the backup (SSH).
   Port	The port to which to connect (SSH).
   Directory	The directory into which to write the backup on the remote SSH server. Backups are written into a subdirectory called YYYY-MM-DD_hhmmss_addm_backup _machine_UUID_ inside the specified directory. (SSH only).
   Path	The share name and directory name into which to write the backup on a Windows share. Backups are written into a subdirectory called YYYY-MM-DD_hhmmss_addm_backup _machine_UUID_ inside the specified directory. Path syntax is \\sharename\directoryname, where sharename is the name of the Windows share, and directoryname is the name of the directory into which to write the backup. (Windows share only).
   Username	The username to use to connect to the remote server (SSH and Windows share).
   Password	The corresponding password (SSH and Windows share).
   Options
   Verify backup	Select Verify backup to verify (md5) that the files in the backup archive are the same as those on the appliance. We recommend that you do so, particularly for Windows share backups. See Windows share (SMB) backups for more information. Shown only after successfully testing the connection for SSH and Windows share backups.
   Include sensitive data	Select Include sensitive data to include sensitive data with the backup. This includes the vault and the appliance key and certificate. Appliance UI users are always backed up and restored, regardless of this setting. So, for example, after a restore the password in effect for the system user will be the one from the source appliance. Shown only after successfully testing the connection for SSH and Windows share backups.
   Encrypt Backup	Select Encrypt Backup to encrypt the backup. When encryption is disabled the backup does not include security data used by the RESTful API. Without this data, existing API access tokens will not work with the restored system and must be reissued.
   Encryption Passphrase	Enter the passphrase to be used to encrypt the backup. You need to confirm the passphrase.
   Email when complete	Select Email when complete and enter an email address if you want an email to be sent automatically when the backup task is completed. Shown only after successfully testing the connection for SSH and Windows share backups.
   Reduce backup size (slower to restore)	Select Reduce backup size to make the backup files smaller. The backups are faster and smaller using the Reduce backup size option as the datastore indexes are removed. They do however take longer to restore as the indexes must be recreated.
   Test Connection	Click Test Connection to ensure that the remote host can be contacted and that the credentials are valid. Shown only for SSH and Windows share backups.

4. Click Shutdown & Backup to start the cluster backup operation.
   You are prompted for confirmation.
5. Click No to return to the Appliance Backup page. Click Yes to continue and backup the appliance.
   All services are shut down on all members of the cluster for the backup and restarted when the backup is complete. During the backup, a progress window is displayed. A Cancel button is also displayed, but only on the UI for the machine from which the backup was started. The Cancel button is only enabled at the stages of the backup where it is possible to cancel.

To restore a backup to the cluster

1. From the Appliance section of the Administration tab, click Backup & Restore.
   The Appliance Backup page is displayed.

2. Click the Restore Backup tab.
   The Restore Backup tab has a panel in which you can choose the source of the backup, and provides details of the size and contents of the existing local backup.
   

3. Enter the details for the backup source.

   The fields that can be completed are displayed or hidden depending on the backup type selection. Required fields are indicated with a red asterisk.

   Field Name	Details
   Backup Type	Select the source type from the drop-down list. This can be one of the following: • On Members—The backup for each machine is read from that machine's $TIDEWAY/var/backup directory. • SSH—The backup is read from a remote server over ssh. • Windows Share—The backup is read from a Windows share.
   Host	The hostname of the remote server from which to read the backup (SSH).
   Port	The port to which to connect (SSH).
   Directory	The directory from which to read the backup on the remote server (SSH). Backups are written into a subdirectory called YYYY-MM-DD_hhmmss_addm_backup _machine_UUID_ inside the specified directory. Ensure you specify the subdirectory name too. (SSH only).
   Path	The share name and directory name from which to read the backup on the remote server (Windows share). Backups are written into a subdirectory called YYYY-MM-DD_hhmmss_addm_backup _machine_UUID_ inside the specified directory. Path syntax is \\sharename\directoryname, where sharename is the name of the Windows share, and directoryname is the name of the directory containing the backup. (Windows share only).
   Username	The username to use to connect to the remote server (SSH and Windows share).
   Password	The corresponding password (SSH and Windows share).
   Preserve Identity	Select Take Appliance Identity from the backup or Preserve current Appliance Identity from the drop-down. Appliance identity consists of: • Appliance identity • HTTPS configuration • Consolidation configuration
   Email when complete	Select the check box and enter an email address if you want an email to be sent automatically when restoring the backup is complete.
   Test Connection	When you enter valid connection information, the Test Connection button is enabled. Click this to test the connection to the remote backup server. If the test is successful, and a backup is present, the Remote Backup Details pane displays information on the remote backup.

4. Click Shutdown & Restore to start the restore operation.
   You are prompted for confirmation.
5. Click No to return to the Appliance Backup page. Click Yes to continue and restore the cluster.
   All services are shut down on all members of the cluster for the restore and restarted when the restore is complete.

Accessing contents of an encrypted backup

You can access the contents of an encrypted backup on a cluster in the same way as for a standalone appliance. The backup held on each member is the backup for that member only; not for the cluster.