Archiving and restoring logs
By default, logs are retained for specific number of days (depending on your license entitlements) and are available for analysis. To retain the logs for a longer duration for on-demand analysis, compliance, or other purposes, as an administrator, archive the logs and move them to cold storage.
Life cycle of logs
The following important terms will help you understand the life cycle of the collected logs:
- Retention period—Time for which logs are available for analysis, after which they are archived. During this period, the logs are saved in the hot storage where they are indexed for search. You can analyze and search the logs. When the retention period is over, logs are moved to the cold storage or are archived. At the same time, the search indexes are also removed from these logs that means you cannot access the archived logs on the Discover page.
- Archive period—Time period for which logs are retained and archived, after which they are purged. During this period, logs are stored in the cold storage and search indexes are removed. Logs are archived for the time defined in the archive period that starts from the log collection date. Therefore, you cannot access the archived logs on the Discover page.
When the archive period is over, logs are purged and cannot be restored. - Restore period—Time period for which logs are restored for analyzing, after which they are autoarchived. When the logs are archived, you can restore them for analysis. Restored logs are available in the Explorer for analysis. After the restore period is over, logs are archived automatically. However, you can archive the restored logs manually to free up the space to restore other logs if the maximum data storage limit is reached.
The retention, archive, and restore period depend upon your license entitlement and are shown to you on the Configurations > Archive & Restore page.
The following image illustrates logs life cycle:
Watch this video (2:56) to understand the archiving and restoring feature.
Archiving and Restoring Logs in BMC Helix Log Analytics.mp4
Changes to index pattern after archiving is enabled
Logs are archived in a log index with a unique name. 100 GB data is stored in an index. If more than 100 GB data is collected in a tenant in a day, multiple indices are created for a particular date. You can identify the archived data by viewing the Date column that shows the date on which the logs were collected. You require this date to determine which log index you want to restore.
After archiving is enabled, the format of the index pattern changes to logarc_*. All logs are collected and shown in this index pattern. Archived and restored data are also available in the preceding index pattern. Therefore, to analyze logs collected after archiving is enabled, use the logarc_* index pattern.
To restore archived logs
- To search the logs you want to restore, use the Date column or the Search Data field.
- Select the check box for the archived logs index, and select Actions > Restore.
- To restore multiple archived logs, select the check boxes for those log indices and click the Restore button.
Automatic archiving of logs
Logs that you restore remain available for analysis for the restore period. When the restore period is over, logs are archived automatically. The number of days in which the logs will be autoarchived are shown in the Autoarchived Days column. This value is also based on your license entitlement.
For example, the restore period for your tenant is 5 days. You restored a log index on June 1st. This log index will be autoarchived on June 6th.
In some cases, the autoarchived period can be less than the restore period for your tenant. This occurs because the archive period for the particular log index gets over before the restore period. For example, you restored a log index on June 1st. The restore period is 5 days and it should be autoarchived in 5 days, on June 6th. However, you see that 3 is showing in the Autoarchived Days column. It is because the log index that you restored is due to be purged in 3 days, on June 4th.
To archive restored logs manually
- To search the logs that you want to archive, use the Date column or the Search Data field.
- Select the check box for the restored logs, and select Actions > Archive.
- To archive multiple restored logs, select the check boxes for those log indices and click the Archive button.
- To verify that the logs are archived, on the Discover page, in the logarc_* index pattern, search for the logs in the time range that you just archived.
If no results are shown, logs are archived correctly.
Where to go from here