tw_ds_offline_compact

Over time, the database files within the datastore can become fragmented, meaning that the data within them is structured inefficiently, so the files take up an unnecessarily large amount of disk space, and data access speed can suffer. The tw_ds_offline_compact utility enables you to compact the datastore by writing new copies of the database files. As it creates new files, the data is packed more efficiently, helping to alleviate lost disc space caused by fragmentation.

The tool operates on an offline system, meaning that the tideway services must be stopped when you use this tool.

Compacting the datastore might take a long time. In a large datastore, this can be many hours. Once a compaction starts, you should not interrupt it. To prevent loss of terminal connection interrupting the compaction, you should run tw_ds_offline_compact inside the screen utility which is installed on the appliance. The user example below shows how to do this.

To use the utility on a cluster, ideally you should stop the tideway services on all machines in the cluster and run tw_ds_offline_compact on each machine and then restart the tideway services when all compactions have completed.This method has the advantage that all machines can be compacted at the same time and the elapsed time is minimized. Alternatively, if you are running a cluster of three or more with fault tolerance enabled, and cannot stop your cluster, you could stop the tideway services on a single machine, run tw_ds_offline_compact on that machine, and then restart the tideway services. Once the machine is working again in the cluster, you can perform the compaction on the remaining machines in turn. This method has disadvantages, compactions run individually in turn, so the elapsed time spent in compaction is much greater. Additionally, once a machine has been compacted, it needs to catch up with the activity that it missed while offline. In a busy cluster, this could take considerable time, depending on the cluster size, datastore size, and performance.

To use the utility, enter the following command:

tw_ds_offline_compact [options]

where options are any of the options described in the following table and the common command line options described in Using command line utilities.

--smallest-first

User example

The following example shows backing up a datastore and then compacting it using the tw_ds_offline_compact utility.

Backup the datastore

Back up the datastore using the appliance backup tool.

Compact the datastore

1. Login to the appliance command line interface as the tideway user.

2. Run the screen utility. Enter:

   [tideway@appliance01 ~]$ screen

3. Stop the tideway services.

   [tideway@appliance01 ~]$ tw_service_control --stop

4. Run the tw_ds_offline_compactutility. Enter:

   [tideway@appliance01 ~]$ tw_ds_offline_compact

   The utility checks that the tideway services have been stopped and that there is sufficient disk space to continue. It then reports the data directory, largest database file, and the free space available. Before proceeding you must confirm that you have made a backup of the datastore.

5. Enter yes to confirm that you have made a backup of the datastore.
   You are then prompted to confirm that you want to start the compaction.
6. Enter yes to start the compaction.
   The utility starts and reports progress until completion.
   Do not interrupt the process.

7. Start the tideway services. Enter the following command:

   [tideway@appliance01 ~]$ tw_service_control --start

Recovering from a lost connection using screen

If you lose the connection to the appliance and you have used screen, you can reconnect to the appliance and recover the virtual terminal running the compaction. To do this:

1. Reconnect to the appliance and login as the tideway user.

2. List the current screen sessions. Enter:

   [tideway@appliance01 ~]$ screen -ls
   There is a screen on:
           23274.pts-0.appliance01        (Detached)
   1 Socket in /var/run/screen/S-tideway.
   

   
   If there is only one screen listed, you can re-attach to it with a simple command:

   [tideway@appliance01 ~]$ screen -r

   
   If there is more than one, copy the screen identifier:

   [tideway@appliance01 ~]$ screen -r 23274.pts-0.appliance01

   
   The virtual terminal is recovered and you can see how the compaction is progressing until completion.

3. Start the tideway services. Enter the following command:

   [tideway@appliance01 ~]$ tw_service_control --start

Recovering from an interrupted compaction

If the compaction has been interrupted in some way, then the database files are left in a partial state and the datastore cannot run. You can recover from this situation in the following ways:

• Perform the compaction using the tw_ds_offline_compact utility again and allow it to complete without interruption. See the procedure above.
• Run the tw_ds_offline_compact utility again with the --fix-interrupted option. This fixes the datastore but does not perform any more compaction.