Archiving

You can maintain system performance and comply with record retention policies by using the Archiving feature available in BMC Helix ITSM. The Archiving process helps you to achieve these goals:

• Remove obsolete records from your production database by using a regular, controlled, and predictable process. 
• Move qualified transactional data records from the related production data forms including certain associated forms to a set of corresponding archiving forms, all within the same database.

Lifecycle of an archived record

The following video illustrates how records cycle through the archiving process (viewing time is 3:56).

http://youtube.com/watch?v=CYKWZhP5Tns?rel=0

Overview of the archiving process

By default, the archiving process runs every 24 hours and uses a set of archiving policies to determine when a record is ready to be moved to the archive. The archiving policy defines:

• Which record types are archived (Incident requests, change requests, and so on).

• The required record status (Closed, Retired, or Cancelled).

  Important

  The archiving process does not consider when the record was Closed, Retired, or Cancelled. For example, an incident request that was submitted 548 days ago, but was closed only yesterday, is archived during the same archiving run as an incident request that was submitted 548 days ago and closed on the same day. If an incident request was submitted 600 days ago, but closed only yesterday, it is archived in the first archiving run after it is closed.

• When to archive a record, according to the Age in Days value (a reference to how long ago a record was either submitted or last modified), depends on the record type.

You can control the archiving process by updating individual archiving policies to meet the needs of your organization. For example, you can archive records of a specific type more or less frequently by providing a custom value for the Age in Days field, which controls when the record is ready for archiving. You can also disable any given archiving policy to prevent the associated record type from being archived, or turn the entire archiving process off and on.

When an archiving process takes place, it copies the qualified transactional records from the production form to a corresponding archive form, and then deletes the record from the production form.

Archiving implemented from AR System

Although the archiving process is available with the BMC Helix ITSM suite of applications and helps to support the goals of improving application performance, it is implemented at the platform level, through AR System. 

For information about how AR System implements the archiving process see   Archiving data in the BMC AR System online documentation.

Archiving architecture

The archiving process makes use of the form-level association management capabilities. By using association management, you can track and maintain the relationships among the BMC Helix ITSM application forms.

Association management in AR System

 For information about using associations, see Defining data associations among forms . 

Archiving benefits

Archiving data regularly provides the following benefits: 

• Reduces the size of your production data sets
• Improves overall system performance. For example, searches run more quickly, because they look only at production data and not at archived data
• Enforces organizational data-retention policies

Online conversation about archiving benefits

https://www.youtube.com/mVL8LgIZG54?rel=0

Archiving FAQs

The following FAQs provide you with information about the archiving process: 

How do I implement archiving?

When you install or upgrade to BMC Helix ITSM 9.0 or later, archiving is enabled by default for all out-of-the-box forms. 

What if my installation uses custom forms?

If you perform customizations in your BMC Helix ITSM environment, you should be aware of the archiving information in the AR System online documentation. The information in the AR System documentation is intended for people who need a deeper understanding of how the archiving process works, for example, people who create custom forms. In the AR System online documentation, start with the  Archiving data  topic. 

In general, if your installation has custom forms that are child forms of a primary application form (for example, your custom form is a child of the HPD:Help Desk form), you must define an association to follow between the primary application form and the custom child form. See Configuring data archiving associations for related forms  the AR System online documentation for information about creating an association between a primary form and a child. For more information about associations, see Defining data associations among forms . 

If you create new primary forms for which you want to enable archiving, you must enable form level archiving on that form. See  Configuring data archiving for a form  in the AR System online documentation for information about enabling form level archiving. 

If upgrading, can earlier customizations affect archiving?

If you are upgrading, customizations performed on an earlier installation can prevent some records from being archived.

To ensure that previous customization won't affect archiving, review all custom and overlaid filters that are both:

• Defined for parent and child forms covered by the archiving policies
• Triggered by an On Delete 

You must ensure that these filters do not cause exceptions during the archiving process. See also What happens if the archiving process encounters an error or exception condition in a record?

After an upgrade, can I archive records that were created before the upgrade?

Yes. All records that meet the criteria defined in the archiving process policy for that record type are archived, regardless of when they were submitted. To read more about archiving process policies and how to manage them, see Managing the Archiving process policies.

Which application records are archived?

• BMC Change Management
• BMC Knowledge Management
• BMC Service Desk
• BMC Service Request Management
• Approval
• Task
• AR System email messages

What types of records are not archived?

The following record types are not archived:

• BMC Requester console records
• Foundation data
• Process setup 

How can I find out which associated forms are archived with the main forms?

If you have Archive Admin permissions, you can see a list of the associated forms that are archived along with the main form by opening the main form in Developer Studio (for example, HPD:Helpdesk) and then opening the Definitions tab. On the Definitions tab, expand the Other Definitions panel and then expand the Associations to Follow for Archive panel.

How does the archiving process determine when a record is ready to archive?

It depends on the application. Refer to the following sections for a description of when the given main forms are archived for each BMC Helix ITSM application:

BMC Change Management

By default, the archiving process archives BMC Helix ITSM: Change Management records according to the following criteria (all of the conditions must be true):

• The record status must be Closed
  In some cases, however, child records such as Task, which are still open, will be archived.
• The Do Not Archive  flag value for the record must be null; which means, the flag is not selected.
• The number of days elapsed since the record Submit Date must be at least 548 (18 months). This value represents the default value and is configurable for each main form by changing the Age in days value.

BMC Knowledge Management

By default, the archiving process archives the following BMC Helix ITSM: Knowledge Management record types:

• Knowledge Management
• Search History
• Search History Company
• Search History Operational Categorization
• Search History Organizational Department
• Search History Product Categorization
• Search History Service
• Search History Site Region
• Search History Source
• Search History Visibility Group

The archiving process archives BMC Helix ITSM: Knowledge Management records according to the following criteria (all of the conditions must be true):

• The record status must be one of the following values depending on the record type:
  • Retired
  • Cancelled
  • Closed Version
• The  Do Not Archive  flag value for the record must be null (that is, the flag is not selected).
• The number of days elapsed since the record's last modified date must be at least 90 (3 months). This value represents the default value and is configurable for each main form by changing the Age in days  value.

BMC Service Desk

By default, the archiving process archives the following BMC Helix ITSM: Service Desk record types:

• Incident Management
• Problem Management
• Solution Database
• Known Error

The archiving process archives Service Desk records according to the following criteria (all of the conditions must be true): 

• The record status must be Closed
  In some cases, however, child records such as Task, which are still open, will be archived.
• The  Do Not Archive  flag value for the record must be null; that is, the flag is not selected.
• The number of days elapsed since the record Submit Date must be at least 548 (18 months). This value represents the default value and is configurable for each main form by changing the Age in days  value.

BMC Service Request Management

By default, the archiving process archives the following BMC Service Request Management record types:

• Service requests
• Work orders

The Archive policy archives BMC Service Request Management records according to the following criteria (all of the conditions must be true):

• The record status must be one of the following values:
  • Cancelled
  • Closed 
• The  Do Not Archive  flag value for the record must be null; that is, the flag is not selected.
• The number of days elapsed since the record Submit Date  must be at least 548 (18 months). This value represents the default value and is configurable for each main form.

If my environment uses data mirroring, will that have an impact on the archiving process?

No. Data mirroring and the Archive process can happen simultaneously without one operation interfering with the other.

How do I administer the archiving process?

By using the AR System Archive Manager console, you can administer the archiving policies that tell the system when to move records from the production forms to the archive forms. You can also use the AR System Archive Manager console to delete all older archived records. For more information about administering archiving policies from the AR System Archive Manager console, see Managing the Archiving process policies.

How often are records archived?

The archiving process uses a concept called continuous archiving. Continuous archiving employs a mechanism that automatically archives records, by default, every 24 hours (you can configure the archiving process process to run more or less often, according to the needs of your environment). Continuously archiving records limits the strain that an archive run has on the system. By limiting the strain on the system, you help to ensure performance and reliability compared to archiving a larger volume of records, for example, every three months.

Can I stop an archiving run when it is in mid-process?

Yes. There are a couple of ways to stop an archiving run while it is in mid-process, depending on what you need to stop:

• Scheduled, full archive run—To stop an entire, scheduled archive run in mid-process, from the AR System Archive Manager console, turn archiving off. 
• Individual archiving policy—To stop an individual archiving policy in mid-process when it is part of a scheduled archiving run, disable the policy in the AR System Archive Manager console. When you disable an individual policy that is part of a regular archiving run, the archiving run skips the remainder of the qualified records covered by the disabled policy and moves on to the next archive policy.

• On-demand archive—To stop an on-demand archive, disable the applicable policy in the AR System Archive Manager console.

• When you turn off archiving or disable an archiving policy as mentioned here, the archiving process finishes archiving the record currently being processed and then either stops or moves on to the next archiving policy. All of the records that the archiving run processed before you disabled it remain committed to the Archive form.

What happens if the archiving process encounters an error or exception condition in a record?

The main form and all related child forms of a given record must be error or exception free. If the archiving process encounters any errors or exceptions in a record, it rolls back the entire archive process for that record and does not include the record in that archiving run.

For information about troubleshooting records that meet the archiving policy but are not being picked up in the archiving process, click here.

Can you search for archived data from the BMC Helix ITSM applications?

Because archived records are removed from the production system and stored in an archive form, you cannot find them from the application by searching. However, you can search for archived data by opening the archive form. For example, you can search an archived change request by opening the CHG:Infrastructure_Change_Archive form. If searching is not enabled in your archive form, you must enable web toolbar to get the search option. For information about enabling web toolbar, see  Customizing a standard or web view

Does archiving work differently in a multitenant environment? 

No. The archiving policies are tied to main application forms, not tenancy. You can define only one archiving policy for each main application form. For example, if your multitenant environment has three companies, Company A, Company B, and Company C, the archiving process applies the one Incident Management archiving policy equally to the incident requests in each of those companies. The qualified incident requests from companies A, B, and C are all archived in the same run of the archiving process.

What happens to the permissions of archived records?

Archiving has no impact on permissions. When a record is moved from a production form to an archiving form, the archiving form inherits the permission levels of the production form.

What happens to child records when the parent record is archived?

When a parent record is archived, all of its child records are also archived, regardless of their state. For example, when an incident record with three associated tasks (one of which still has a status of Open) is archived, all of the task records are archived, too, including the task that is still open. In addition, any broadcasts created directly from an archived record are removed from the production system.

When two peer records are related, what happens when one is archived and the other is not?

Consider the following scenario: You have a change request, which according to the archiving policy is ready for archiving and this change request is related to an incident request that is not ready for archiving.

After the next archiving process runs:

• The change request is archived.
• The incident request is still available in the production environment.
• On the incident request, the Relationships table on the Relationship tab still contains a reference to the archived change request, but if you select the reference in the table the system displays a message telling you that the selected record was archived. 
• The Quick Actions menu of the Relationships table is disabled while the archived record shown in the table is selected.

Does archiving affect sample transactional data?

Yes. Sample transactional data will be archived if it meets the archiving criteria. To prevent seed and sample data from being archived, use the  Do Not Archive flag on that data.  

Can an application access data on the archive forms?

No. The archive forms have no code associated with them. The archive forms are only data repositories organized to look like the production form. Because the archiving process is meant to help maintain system performance by eliminating obsolete records from the production forms, after data is copied to an archive form, it cannot be copied back into production.

Who can access the data after it is archived, and how?

System administrators can access archived data by generating reports that can be run against the archiving forms, using third party reporting applications.

Is there any impact on overall system performance when the Archive process is running?

No. It has no impact on overall system performance. See the Multi-threaded archiving in the AR System server section of   19.02 enhancements  in AR Server documentation.

Are there other ways to maintain system performance?

Yes. Make sure that you continue to perform regular database maintenance activities to help maintain the performance gains realized by running the archiving process.

Are there ways to use the archiving process to improve system performance without creating an archive?

Yes. Using BMC Developer Studio, you can change the Archive Type value (located on the Archiving tab) of the production forms to Delete From Source (the default value is Copy to Archive and Delete from Source). If you change the value to Delete From Source, when the Archive process runs, it deletes the records from the production forms that meet the archiving policy without also writing them to the archive forms. You must make this change for each of the main application forms and their associated child forms (and child forms of child forms, and so on). This change controls the size of your production database, which improves overall system performance, without also creating an archive.

Because you must be in Base Development Mode to do this work, you cannot overlay these changes. This restriction means that when you upgrade your installation, you must repeat the process of changing the Archive Type to Delete from Source for all of the main forms and their associated child forms.

How to archive attachments?

The No attachment flag in the Archive Property section of a form determines whether to archive the attachments or not. To archive attachments, make sure that the flag is not selected. For more information see  Configuring data archiving for a form .