Purging soft-deleted data to optimize CMDB performance
The Normalization Engine and the Reconciliation Engine enable you to normalize CIs across data sets and then reconcile the duplicate CIs into a final production data set. Besides this, it is equally important to regularly purge the soft-deleted data by using Reconciliation. This helps eliminate CIs that are no longer required in your environment.
Use a Purge activity to physically delete only instances that have been marked as deleted, or soft-deleted. You can optionally further restrict the instances to be purged by using a Qualification Set.
Best practice
If you are purging data from data sets that also serve as source data sets in a Merge activity, you can choose to only delete instances that have also been soft deleted in the target data set. This helps prevent "orphan" instances in the target data set that would never get marked as deleted because no source ever merges with them.
To ensure that users have a chance to review soft-deleted records, we recommend that you make Purge activities part of a standalone job that is executed manually or by workflow events.
To create a Purge reconciliation activity
- From CMDB Portal, select Jobs > Manage Reconciliation.
- Click Create Job and enter a job name.
- Select the appropriate settings for the job.
To view the details of every setting, hover the cursor over each setting. - Click Add Activity > Purge and perform the following steps:
Failed to execute the [excerpt-include] macro. Cause: [Error number 2 in 0: No wiki with id [confluencePage:page] could be found]. Click on this message for details.
org.xwiki.rendering.macro.MacroExecutionException: Failed to get document for reference [confluencePage:page:ac221.Standard reconciliation jobs]
at com.xwiki.macros.excerptinclude.internal.macro.ExcerptIncludeMacro.internalExecute(ExcerptIncludeMacro.java:130)
at productHelper.macros.BmcExcerptIncludeMacro.internalExecute(BmcExcerptIncludeMacro.java:27)
at productHelper.macros.BmcExcerptIncludeMacro.internalExecute(BmcExcerptIncludeMacro.java:18)
at com.xwiki.macros.AbstractProMacro.execute(AbstractProMacro.java:116)
at org.xwiki.rendering.internal.transformation.macro.MacroTransformation.transform(MacroTransformation.java:441)
at org.xwiki.rendering.internal.transformation.DefaultRenderingContext.transformInContext(DefaultRenderingContext.java:183)
at org.xwiki.rendering.internal.transformation.DefaultTransformationManager.performTransformations(DefaultTransformationManager.java:88)
at org.xwiki.display.internal.DocumentContentAsyncExecutor.executeInCurrentExecutionContext(DocumentContentAsyncExecutor.java:396)
at org.xwiki.display.internal.DocumentContentAsyncExecutor.execute(DocumentContentAsyncExecutor.java:269)
at org.xwiki.display.internal.DocumentContentAsyncRenderer.execute(DocumentContentAsyncRenderer.java:112)
at org.xwiki.rendering.async.internal.block.AbstractBlockAsyncRenderer.render(AbstractBlockAsyncRenderer.java:157)
at org.xwiki.rendering.async.internal.block.AbstractBlockAsyncRenderer.render(AbstractBlockAsyncRenderer.java:54)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.syncRender(DefaultAsyncRendererExecutor.java:290)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.render(DefaultAsyncRendererExecutor.java:267)
at org.xwiki.rendering.async.internal.block.DefaultBlockAsyncRendererExecutor.execute(DefaultBlockAsyncRendererExecutor.java:125)
at org.xwiki.display.internal.DocumentContentDisplayer.display(DocumentContentDisplayer.java:67)
at org.xwiki.display.internal.DocumentContentDisplayer.display(DocumentContentDisplayer.java:43)
at org.xwiki.display.internal.DefaultDocumentDisplayer.display(DefaultDocumentDisplayer.java:96)
at org.xwiki.display.internal.DefaultDocumentDisplayer.display(DefaultDocumentDisplayer.java:39)
at org.xwiki.sheet.internal.SheetDocumentDisplayer.display(SheetDocumentDisplayer.java:123)
at org.xwiki.sheet.internal.SheetDocumentDisplayer.display(SheetDocumentDisplayer.java:52)
at org.xwiki.display.internal.ConfiguredDocumentDisplayer.display(ConfiguredDocumentDisplayer.java:68)
at org.xwiki.display.internal.ConfiguredDocumentDisplayer.display(ConfiguredDocumentDisplayer.java:42)
at com.xpn.xwiki.doc.XWikiDocument.display(XWikiDocument.java:1412)
at com.xpn.xwiki.doc.XWikiDocument.getRenderedContent(XWikiDocument.java:1548)
at com.xpn.xwiki.doc.XWikiDocument.displayDocument(XWikiDocument.java:1498)
at com.xpn.xwiki.doc.XWikiDocument.displayDocument(XWikiDocument.java:1467)
at com.xpn.xwiki.api.Document.displayDocument(Document.java:788)
at jdk.internal.reflect.GeneratedMethodAccessor574.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:571)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:554)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:221)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:368)
at org.apache.velocity.runtime.parser.node.ASTReference.value(ASTReference.java:704)
at org.apache.velocity.runtime.parser.node.ASTExpression.value(ASTExpression.java:75)
at org.apache.velocity.runtime.parser.node.ASTSetDirective.render(ASTSetDirective.java:242)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:147)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:439)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:190)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:147)
at org.xwiki.velocity.internal.directive.TryCatchDirective.render(TryCatchDirective.java:86)
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:304)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:439)
at org.apache.velocity.Template.merge(Template.java:358)
at org.apache.velocity.Template.merge(Template.java:262)
at org.xwiki.velocity.internal.InternalVelocityEngine.evaluate(InternalVelocityEngine.java:225)
at com.xpn.xwiki.internal.template.VelocityTemplateEvaluator.evaluateContent(VelocityTemplateEvaluator.java:105)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.evaluateContent(TemplateAsyncRenderer.java:219)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.renderVelocity(TemplateAsyncRenderer.java:174)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.render(TemplateAsyncRenderer.java:135)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.render(TemplateAsyncRenderer.java:54)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.lambda$syncRender$0(DefaultAsyncRendererExecutor.java:284)
at com.xpn.xwiki.internal.security.authorization.DefaultAuthorExecutor.call(DefaultAuthorExecutor.java:98)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.syncRender(DefaultAsyncRendererExecutor.java:284)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.render(DefaultAsyncRendererExecutor.java:267)
at org.xwiki.rendering.async.internal.block.DefaultBlockAsyncRendererExecutor.render(DefaultBlockAsyncRendererExecutor.java:154)
at com.xpn.xwiki.internal.template.InternalTemplateManager.render(InternalTemplateManager.java:904)
at com.xpn.xwiki.internal.template.InternalTemplateManager.renderFromSkin(InternalTemplateManager.java:866)
at com.xpn.xwiki.internal.template.InternalTemplateManager.render(InternalTemplateManager.java:853)
at com.xpn.xwiki.internal.template.InternalTemplateManager.renderNoException(InternalTemplateManager.java:808)
at com.xpn.xwiki.internal.template.InternalTemplateManager.renderNoException(InternalTemplateManager.java:800)
at com.xpn.xwiki.internal.template.DefaultTemplateManager.renderNoException(DefaultTemplateManager.java:79)
at com.xpn.xwiki.internal.template.DefaultTemplateManager.renderNoException(DefaultTemplateManager.java:73)
at org.xwiki.template.script.TemplateScriptService.render(TemplateScriptService.java:54)
at jdk.internal.reflect.GeneratedMethodAccessor18423.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:571)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:554)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:221)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:368)
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:492)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:147)
at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:218)
at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:331)
at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:261)
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:304)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:439)
at org.apache.velocity.Template.merge(Template.java:358)
at org.apache.velocity.Template.merge(Template.java:262)
at org.xwiki.velocity.internal.InternalVelocityEngine.evaluate(InternalVelocityEngine.java:225)
at com.xpn.xwiki.internal.template.VelocityTemplateEvaluator.evaluateContent(VelocityTemplateEvaluator.java:105)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.evaluateContent(TemplateAsyncRenderer.java:219)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.renderVelocity(TemplateAsyncRenderer.java:174)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.render(TemplateAsyncRenderer.java:135)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.render(TemplateAsyncRenderer.java:54)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.lambda$syncRender$0(DefaultAsyncRendererExecutor.java:284)
at com.xpn.xwiki.internal.security.authorization.DefaultAuthorExecutor.call(DefaultAuthorExecutor.java:98)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.syncRender(DefaultAsyncRendererExecutor.java:284)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.render(DefaultAsyncRendererExecutor.java:267)
at org.xwiki.rendering.async.internal.block.DefaultBlockAsyncRendererExecutor.render(DefaultBlockAsyncRendererExecutor.java:154)
at com.xpn.xwiki.internal.template.InternalTemplateManager.render(InternalTemplateManager.java:904)
at com.xpn.xwiki.internal.template.InternalTemplateManager.renderFromSkin(InternalTemplateManager.java:866)
at com.xpn.xwiki.internal.template.InternalTemplateManager.render(InternalTemplateManager.java:853)
at com.xpn.xwiki.internal.template.InternalTemplateManager.renderNoException(InternalTemplateManager.java:808)
at com.xpn.xwiki.internal.template.InternalTemplateManager.renderNoException(InternalTemplateManager.java:800)
at com.xpn.xwiki.internal.template.DefaultTemplateManager.renderNoException(DefaultTemplateManager.java:79)
at com.xpn.xwiki.internal.template.DefaultTemplateManager.renderNoException(DefaultTemplateManager.java:73)
at org.xwiki.template.script.TemplateScriptService.render(TemplateScriptService.java:54)
at jdk.internal.reflect.GeneratedMethodAccessor18423.invoke(Unknown Source)
at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.base/java.lang.reflect.Method.invoke(Method.java:569)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.doInvoke(UberspectImpl.java:571)
at org.apache.velocity.util.introspection.UberspectImpl$VelMethodImpl.invoke(UberspectImpl.java:554)
at org.apache.velocity.runtime.parser.node.ASTMethod.execute(ASTMethod.java:221)
at org.apache.velocity.runtime.parser.node.ASTReference.execute(ASTReference.java:368)
at org.apache.velocity.runtime.parser.node.ASTReference.render(ASTReference.java:492)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:147)
at org.apache.velocity.runtime.directive.VelocimacroProxy.render(VelocimacroProxy.java:218)
at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:331)
at org.apache.velocity.runtime.directive.RuntimeMacro.render(RuntimeMacro.java:261)
at org.apache.velocity.runtime.parser.node.ASTDirective.render(ASTDirective.java:304)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:147)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:171)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:147)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:439)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:190)
at org.apache.velocity.runtime.parser.node.ASTBlock.render(ASTBlock.java:147)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:439)
at org.apache.velocity.runtime.parser.node.ASTIfStatement.render(ASTIfStatement.java:190)
at org.apache.velocity.runtime.parser.node.SimpleNode.render(SimpleNode.java:439)
at org.apache.velocity.Template.merge(Template.java:358)
at org.apache.velocity.Template.merge(Template.java:262)
at org.xwiki.velocity.internal.InternalVelocityEngine.evaluate(InternalVelocityEngine.java:225)
at com.xpn.xwiki.internal.template.VelocityTemplateEvaluator.evaluateContent(VelocityTemplateEvaluator.java:105)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.evaluateContent(TemplateAsyncRenderer.java:219)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.renderVelocity(TemplateAsyncRenderer.java:174)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.render(TemplateAsyncRenderer.java:135)
at com.xpn.xwiki.internal.template.TemplateAsyncRenderer.render(TemplateAsyncRenderer.java:54)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.lambda$syncRender$0(DefaultAsyncRendererExecutor.java:284)
at com.xpn.xwiki.internal.security.authorization.DefaultAuthorExecutor.call(DefaultAuthorExecutor.java:98)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.syncRender(DefaultAsyncRendererExecutor.java:284)
at org.xwiki.rendering.async.internal.DefaultAsyncRendererExecutor.render(DefaultAsyncRendererExecutor.java:267)
at org.xwiki.rendering.async.internal.block.DefaultBlockAsyncRendererExecutor.render(DefaultBlockAsyncRendererExecutor.java:154)
at com.xpn.xwiki.internal.template.InternalTemplateManager.render(InternalTemplateManager.java:904)
at com.xpn.xwiki.internal.template.InternalTemplateManager.renderFromSkin(InternalTemplateManager.java:866)
at com.xpn.xwiki.internal.template.InternalTemplateManager.renderFromSkin(InternalTemplateManager.java:846)
at com.xpn.xwiki.internal.template.InternalTemplateManager.render(InternalTemplateManager.java:832)
at com.xpn.xwiki.internal.template.DefaultTemplateManager.render(DefaultTemplateManager.java:91)
at com.xpn.xwiki.internal.template.DefaultTemplateManager.render(DefaultTemplateManager.java:85)
at com.xpn.xwiki.XWiki.evaluateTemplate(XWiki.java:2564)
at com.xpn.xwiki.web.Utils.parseTemplate(Utils.java:180)
at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:651)
at com.xpn.xwiki.web.XWikiAction.execute(XWikiAction.java:339)
at com.xpn.xwiki.web.LegacyActionServlet.service(LegacyActionServlet.java:108)
at javax.servlet.http.HttpServlet.service(HttpServlet.java:733)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:227)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at com.xpn.xwiki.web.ActionFilter.doFilter(ActionFilter.java:122)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.xwiki.wysiwyg.filter.ConversionFilter.doFilter(ConversionFilter.java:61)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.apache.tomcat.websocket.server.WsFilter.doFilter(WsFilter.java:53)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.xwiki.container.servlet.filters.internal.SetHTTPHeaderFilter.doFilter(SetHTTPHeaderFilter.java:63)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.xwiki.resource.servlet.RoutingFilter.doFilter(RoutingFilter.java:132)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.xwiki.container.servlet.filters.internal.SavedRequestRestorerFilter.doFilter(SavedRequestRestorerFilter.java:208)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.xwiki.container.servlet.filters.internal.SetCharacterEncodingFilter.doFilter(SetCharacterEncodingFilter.java:117)
at org.apache.catalina.core.ApplicationFilterChain.internalDoFilter(ApplicationFilterChain.java:189)
at org.apache.catalina.core.ApplicationFilterChain.doFilter(ApplicationFilterChain.java:162)
at org.apache.catalina.core.StandardWrapperValve.invoke(StandardWrapperValve.java:202)
at org.apache.catalina.core.StandardContextValve.invoke(StandardContextValve.java:97)
at org.apache.catalina.authenticator.AuthenticatorBase.invoke(AuthenticatorBase.java:542)
at org.apache.catalina.core.StandardHostValve.invoke(StandardHostValve.java:143)
at org.apache.catalina.valves.ErrorReportValve.invoke(ErrorReportValve.java:92)
at org.apache.catalina.valves.AbstractAccessLogValve.invoke(AbstractAccessLogValve.java:687)
at org.apache.catalina.core.StandardEngineValve.invoke(StandardEngineValve.java:78)
at org.apache.catalina.valves.RemoteIpValve.invoke(RemoteIpValve.java:764)
at org.apache.catalina.connector.CoyoteAdapter.service(CoyoteAdapter.java:354)
at org.apache.coyote.http11.Http11Processor.service(Http11Processor.java:382)
at org.apache.coyote.AbstractProcessorLight.process(AbstractProcessorLight.java:65)
at org.apache.coyote.AbstractProtocol$ConnectionHandler.process(AbstractProtocol.java:888)
at org.apache.tomcat.util.net.NioEndpoint$SocketProcessor.doRun(NioEndpoint.java:1684)
at org.apache.tomcat.util.net.SocketProcessorBase.run(SocketProcessorBase.java:49)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1136)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:635)
at org.apache.tomcat.util.threads.TaskThread$WrappingRunnable.run(TaskThread.java:61)
at java.base/java.lang.Thread.run(Thread.java:840)
Caused by: com.xpn.xwiki.XWikiException: Error number 3202 in 3: Exception while reading document [confluencePage:page:ac221.Standard reconciliation jobs()]
at com.xpn.xwiki.store.XWikiHibernateStore.loadXWikiDoc(XWikiHibernateStore.java:1233)
at com.xpn.xwiki.store.XWikiCacheStore.loadXWikiDoc(XWikiCacheStore.java:399)
at com.xpn.xwiki.XWiki.getDocument(XWiki.java:2195)
at com.xpn.xwiki.XWiki.getDocument(XWiki.java:2257)
at com.xwiki.macros.excerptinclude.internal.macro.ExcerptIncludeMacro.internalExecute(ExcerptIncludeMacro.java:128)
... 188 more
Caused by: com.xpn.xwiki.XWikiException: Error number 2 in 0: No wiki with id [confluencePage:page] could be found
at com.xpn.xwiki.internal.store.hibernate.HibernateStore.beginTransaction(HibernateStore.java:854)
at com.xpn.xwiki.store.XWikiHibernateBaseStore.beginTransaction(XWikiHibernateBaseStore.java:576)
at com.xpn.xwiki.store.XWikiHibernateStore.loadXWikiDoc(XWikiHibernateStore.java:1082)
... 192 more - (Optional) Select a schedule for the job run frequency.
- Click Save.
- (Optional) To run the job immediately, open the job from All Job Run Activities and click Start Job.
Important
- For a reconciliation job, if you clear the Disable Progress Bar on UI During Job Execution check box, the number of CIs purged by the reconciliation engine is displayed while the job is running. If you select the check box, the progress bar is not displayed while the job is running, resulting in faster job execution.
- A strong relationship, as defined in Common Data Model, can be purged safely without causing any data integrity issues.