Choosing between a regular Compliance Job and a Batch Job
Whenever a Compliance Job is executed, the BMC Server Automation Application Server spawns a worker thread for the processing of each target being queried. The number of threads spawned concurrently is limited by the MaxWorkItemThreads setting on the Application Server. By default, the value is set to 50 threads.
When a Compliance Job is triggered against a target for the first time, it attempts to create a findfiles cache under the staging directory on the target server and then proceeds to run the Compliance Job. This findfiles cache is a simple file that stores target-related asset information used in processing extended objects. Caching improves the efficiency of the Compliance Job, as it eliminates repeated processing of the same files in multiple extended objects.
When jobs are running against a large number of targets (that is, when the number of targets is larger than the maximum thread count setting), the overall throughput is restricted because creation of the findfiles cache is a synchronous activity and takes a significant amount of time. This causes the thread to wait until the findfiles cache is created before continuing with the Compliance Job.
For such large-scale environments, out-of-the-box batch-type Scale Jobs are provided in the Jobs folder. Each of these Batch Jobs consists of a Compliance Job and a separate Network Shell (NSH) Script Job for cache creation. When the Batch Job is executed, it first creates the findfiles cache as an asynchronous activity across all scoped targets. Only after the cache creation job completes, the Batch Job triggers the Compliance Job, which continues to process the targets synchronously. This method of creating the cache concurrently across targets reduces the overall time for execution of compliance analysis.