Running and monitoring data load jobs

You can run and monitor your data load jobs in Job Console.

Before you begin

A job must have a status of Built before you can run it.

To run a job

  1. From the Applications list on the IT Home Page, select Data Management > Job Console. The Job Console opens.
  2. Double-click on the job that you want to run in the Jobs table. The Job window opens.

  3. Click Run.
    The steps for the jobs are initially assigned a status of Queued in the Steps table and, as they run and data is imported into the specified staging forms, the status changes to Completed.
    A notification is automatically sent to the job owner when:

    • A scheduled job has started
    • A step has a Wait status
    • A step has an error
    • A job is completed

    Tip

    If the job runs without errors, the Job Result drop-down list in the Job window displays Successful.  If there are errors, the Job Result drop-down list displays Errors Detected. These results are also displayed in the Jobs table in the Job Console.
    The Job window also displays the number of steps that have completed out of the total numbers of Load, Validate, and Promote steps in the Completion status field, as shown below.

    udm_completion_status.png
    If errors do occur during the data load, you can view or save these errors by selecting the Load step and clicking View/Save Compressed Error File in the Step window.

After you have run the jobs initially, you can re-run a selected job from the Job window as follows:

  • To run the same job again after correcting any errors in the Error Management console and to revalidate and promote the fixed records, click Re-Run from Validate. This process does not run the Load step again.

    CI loads do not support Re-run from Validate.

  • To make a copy of the job, click Create as New Job. This action creates an exact copy with a new job ID and enables you to run the Load step again to load new data and also run the Validate and Promote steps. You can then click Run to run it again as a new job.

    Tip

    • To remove the wait flag from a step and enable it to run automatically, you can choose from the following procedures:
      • Go to the Job window, select the step, and click Remove Wait
      • Go to the Job window, and click Continue to remove wait flags from all of the highlighted steps
      • Double-click on the step and from the Step window click Continue Step
    • To view the staging form data after the job has run the Load step (and without having to go to the Error Management console), click View Data in the Step window for your Validate or Promote step.

Step run sequence

When a job runs, it takes a copy of the step run sequence created by the Application Administrator in the staging form dependency table and removes any of the sequence elements for which it does not have a matching step. For example, if a job containing only foundation steps runs then the job will keep only the foundation elements from the sequence.

When the Load step occurs, all data is loaded simultaneously.

The Validate and Promote steps for parent staging forms must be completed before child staging forms can be triggered. The child staging forms verify whether the steps for all parent staging forms are completed. If they are then the steps proceed for the child staging forms. If the steps for all parent staging forms are not complete, the child staging forms wait until they are completed. This process repeats itself until the steps for all of the child staging forms are completed.

When each step runs, the status for each step changes to Completed. This status is visible in the Jobs table in the Job Console.

The Promote step and Validate step can occur simultaneously (as long as the staging forms are not parent and child).

In the following example, by the time the Validate step is completed for the People template staging form, the Promote step might have already completed for the Region staging form.

To view the audit log

DMT users can view audit log records for their jobs that provide information about when (date and time) the following events have occurred:

  • A job was created.
  • The job engine re-sequenced the order of steps.
  • A job ran and completed.
  • A user clicked Continue.
  • Job notifications were sent for a job.
  • The Wait state was enabled or removed for a step.
  • A step caused an error.
  • Notifications were sent for a step.
  • A job schedule was created, modified, or deleted.
  • For Error Management, changes were made using Field Maintenance functionality.
  • For Error Management, tree data records were abandoned at the time of automated cleanup (based on application preferences).
  • For staging forms, staging records were abandoned at the time of automated cleanup (based on application preferences).

The following procedure describes how to view the audit log for a job.

  1. In the Job Console, double-click the job for which you want to view the audit log.
  2. In the navigation pane of the Job window, select Links > View Audit Log. The Job Audit Log window is displayed.
  3. Select the Notification Audits tab to view the audit log records.
  4. If you are a DMT Admin, you can select the Execution Audit tab to view the sequence of steps for your job (when jobs started and ended).