Understanding data retention and deletion
Use this topic to understand how the data retention settings function.
After a data collector is created, data collection starts when the first poll happens.
Supposing you want to monitor a file in which data is being continuously added, data starts getting collected from the point when the first poll happened. Note that data older than the creation time is automatically ignored.
The data retention period defines the number of days in the past (from the current date) for which data must be retained in the system. Any data that falls out of the data retention period is automatically removed.
Data retention period has implications on the Read from Past (# days) setting. The Read from Past (#days) setting in the data collector determines the number of days in the past for which data must be read and indexed. The value of this setting must be lesser than or equal to the data retention period. This setting is available for the following data collectors only:
- Upload file used for uploading a file into the system and indexing all the data present in the file.
- Monitor Remote Windows Events used for collecting events generated by a Windows computer.
- Monitor using External Configuration used for collecting event data from external systems integrated with the product.
By default, the product defines the data retention period as seven days. You can change this value for an individual data collector at the time of creating the data collector or for a bulk of data collectors on the Administration > Data Collectors page. The upper limit for setting the maximum data retention limit is set to 14 days, but is configurable. For more information, see Changing system settings.
The data retention period acts as a moving window (depicted in green in the following figure).
Consider that on the following scale of time, you created a data collector at time T1, now data collection starts from T1 when the first poll happens. Data collected at T1 remains in the system until T1+7. As time passes, the data older than the seven days period starts getting deleted and is no longer available for searching.
Comments
Log in or register to comment.