This documentation supports the 9.1 version of Remedy IT Service Management Suite.

To view the latest version, select the version from the Product version menu.

Troubleshooting dataloads

This topic explains the troubleshooting information related to dataloads.

Best Practice

  • If a job fails in the load step, you cannot run that job again. Cancel the job and run this job as a new job.
  • Ensure that the environment is configured correctly for UDM. For example, load balance configurations and data paths.
  • After you run the first UDM job, ensure that the DMT:SYS:SequencingEngine has all the sequence records.
  • Clean up the UDM variable and load forms such as the CTM:LoadPeople records, DMT:ThreadManager, and CAI:Events forms periodically, especially in case of large volumes.
  • Escalations must be enabled and running on your primary sever. For more information, see the how to disable escalations document on BMC Communities..

Troubleshooting load jobs that are stuck in the In Progress state

Issue symptom

Jobs are stuck in the In Progress state.

Issue cause

Escalations are not running on the same pool and are shared with the other running escalations. Additionally, the custom Pentaho job is not configured correctly.

Issue workaround

  1. Ensure that the following escalations are running on the same pool and not shared with the other running escalations:
    1. DMT:DJS:SetStatus—This escalation triggers every one minute and sets the status of the z1D_Action field to CHECKTRANSSTATUS.
    2. Filter DMT:DJS:ChkStepCompleted—This escalation triggers immediately once the z1D_Action field value is set.
      For more information about escalations pools, see Setting escalation pools.
  2. Ensure that the custom Pentaho job is configured correctly. Since the custom Pentaho jobs are considered as customizations, for more information, see the following documentation and BMC Communities pages:

Troubleshooting load data related errors

Issue symptom

The Load Data Errors field in the Step Details information (Step window) for the Load step displays the errors. Click View/Save Compressed Error File to see the errors or save them. 

Additionally, load errors are displayed when Atrium Integrator has encountered an error. Errors range from a systematic Atrium Integrator issue (for example, connectivity issues) to an issue caused when the inbound data is not formatted as required (for example, invalid date formats in the dataload spreadsheets). 

(Click the image to expand it.)

 

Issue cause

When a transformation (containing error handling) has trapped an error for specific data in the Spreadsheet tab or when a required field is missing or you have an incorrect selection of field values in your dataload spreadsheets. Each exception records the row, sheet, and the trapped error message and sends the output to a spreadsheet. Each Spreadsheet tab is included in its own Microsoft Excel file, and, at the end of a job, the files can be zipped up and attached to the load step. 

Issue workaround

Before performing any other troubleshooting procedures, check that the rows in the named spreadsheet have the correct cell formatting applied, as shown in the following list:

  • Row 1—General
  • All other rows—Text

Note

Errors related to incorrect spreadsheet cell formatting are displayed especially if you have copied and pasted information from one spreadsheet to another.

The level of detail provided in the Load Data Errors field is based on the log level specified in the Atrium Integrator jobs and from the Job Console. If you are a DMT administrator, to help you determine why load issues are occurring, alter this value to something more informational (for example, Detailed, Debug, Rowlevel, and so on) so that additional details can be passed back to the step load error.

BMC recommends that if you are using the Debug level:

  • To minimize the volume of data and determine what the warning message is, copy the required job and use only a subset of the data in the spreadsheet.
  • Always reset this level back to its normal value of Minimal to reduce future performance concerns.
  • Ensure that others are not loading data on the computer while you are performing the debugging process. It is easier to work with smaller sets of data when you are identifying issues.

Tips

  • If the load step is stuck in an In Progress state, verify that you have not selected the Disable Escalations server option on the AR System Administration Console under Configuration tab > System > General > Server Information.
  • While running a data load job for a big number of asset records, you must set composite index of the following fields for a faster data load:
    • Company (Attribute added by ITSM extension)
    • Name
    • Serial Number

Troubleshooting Remedy AR System or plug-in errors

Issue symptom

Any Remedy AR System or plug-in related errors that occur at the load step are captured and displayed on the Load step in the Steps panel. The error message contains the details of the error that occurred. 

Issue cause

When such Remedy AR System or plug-in related errors occur, the dataload job fails. Additionally, the load step fails with the Status as Error and a Job Result as Errors Detected.

Issue workaround

The system administrator must resolve such errors and then create a new dataload job. For information about creating a new job, see Creating a job and for information about running a new job, see Running a job.

(Click the image to expand it.)

Troubleshooting unique index violations error

Issue symptom

The ARERR 382 unique index violations error is displayed.

Issue cause

This error usually occurs when you are importing records with ID numbers that are numerically higher than the IDs that are currently assigned by your system.

Issue workaround

For information about the steps to be followed in case of the error, see Importing records with higher ID numbers than those currently issued by system.

Troubleshooting Carte server related errors

Issue symptom

The Load step in the Steps panel captures and displays the errors for the following Carte server issues:

  • The system is unable to establish a connection with the Carte server.
  • The Carte server restarts when a dataload job is run.

Issue workaround

When such errors occur, the Load step is completed with the Status as Completed and Job Result as Errors Detected. As a system administrator, you must resolve such errors and then create a new dataload job.

Troubleshooting common errors

The following table lists the common errors and their workaround.

Issue symptomIssue causeIssue workaround
The load jobs point to the load balancer and a timeout connection error is displayed in the arjavaplugin file.Some parameters are missing from the Microsoft Windows HOST file.

Verify that the load balance server is listed in the UDM:RAppPassword form and the Microsoft Windows HOST file contains the following parameters:

  • AR Server IP servergroupname
  • AR Server IP servergroupname.domain.net

The following error messages are displayed:

No file(s) specified!
Stop processing.

COM_LoadCompany.0 -
Error initializing step

[COM_LoadCompany]
COM_LoadCompanyAlias.0 -
ERROR: No file(s)
specified! Stop processing.

COM_LoadCompanyAlias.0 -
Error initializing step
[COM_LoadCompanyAlias]

The MTATTACHPATH variable is missing from the DMT:TransformationParam form for the involved job.

In a working environment, run the following advanced search on the DMT:TransformationParam form to display the records related to this variable for the AI jobs:

'Variable Name' = "DMTATTACHPATH"

Export and then import the missing records from a working environment to a non-working environment.

The following error message is displayed:

CI-APP-AROutput.0 - Impact String :
couldn't convert String to Integer

CI-APP-AROutput.0 - Unparseable number: " 2000.0"

Issue with a column in a Microsoft Excel spreadsheet.

Verify this issue on the text fields that display a small green triangle. If you do not see this triangle, click in this field to set the integer.

The load step is skipped.A valid Atrium Integrator name is missing from the load step.

Check the step or as the DMT administrator, from the Job Console, check for a valid entry in the Atrium Integrator job list. If no data was loaded for this step or for the related validate and promote steps, ensure that you configure Atrium Integrator. For more information, see Understanding Atrium Integrator.

The following error message is displayed:

Data management process flow is
currently being initialized or
rebuilt, this will finish shortly,
please run the job in a few minutes.

The data management process is getting initialized or restarted.

For more information, see Error when running jobs.

The load step stays the In Progress state even though all the data is on the staging form in a Done state.

The Atrium Integrator engine did not get called.

As an administrator, check that the escalation server is turned on though the Server Information form. Query the UDM:ExecutionStatus form where the execution instance name is the Instance ID of the load step. If you do not find any record, the Atrium Integrator engine (custom AI Job/Transformation) did not get called.

The following error message is displayed:

java.lang.OutOfMemoryError: Java heap space

When there are multiple concurrent jobs running on the Carte server.

Increase the 1GB (default value) of heap space to adopt more concurrent number of jobs.

The following error messages are displayed when performing a search:

Error in plugin" Get List Entry
With Fields not supported on form
UDM:Execution (ARERR 8753) after
Load Balance name change.

Error in plugin : Invalid Execution
Instance. Execution Instance
AGHAA5V0GG40RAN7D2XYEGI6DXP3N5
does not exist or user Remedy
Application Service is not
allowed to access it.

(ARERR 8753) An application
command failed

(ARERR 4554) Application-Delete-Entry
"DMT:Action"

The Microsoft Windows HOST file is not updated.

  • Verify that the server name is correct in the ar.cfg file.
  • Verify that the UDM load path is accessible.
  • Add the following load balance names to the Microsoft Windows HOST file:
    • AR Server IP servergroupname
    • AR Server IP servergroupname.domain.net

The following error messages are displayed:

Failed to initialize at least one step.
Execution can not begin!
No file(s) specified! Stop processing.

Error initializing step
[CTM_LoadPeopleModification

ERROR: Source folder/file [/opt/bmc/ARSystem/db/UDM/DJBnumber]
can not be found!

The UDM load path directory is not shared with the correct permissions.

Verify that the UDM load path is accessible and that the path is a shared path with read and write access.

The Carte server stops responding and the following error message is displayed:

<ERROR> <ARDBCPluginRepository > < ARDBCPluginRepository.java:184 >
createEntry() FAILs in plugin:
ARSYS.ARDBC.PENTAHO

Issues with the Carte server port number.

  1. Verify that the Carte server is running by entering the following URL:
    http://<carte host>:20000
  2. In the Windows Security window, enter the Remedy AR System server user name and password.

From the current state of the jobs running on that Carte server that are displayed, if the server is down, the browser displays a message that the server cannot be connected.

The following exception message is displayed:

Exception in thread "main" java.lang.IllegalArgumentException: Malformed
\uxxxx encoding

The Carte process does not recognize the "\u" entry in the ar.conf file.

For example: BMC Software\ARSystem\ARServer\
Db\userupload.log
.

Correct the path and restart the Remedy AR System server.

The following error message is displayed:

92: "Timeout during database update --
the operation has been accepted
by the server and will usually
complete successfully",
"servername:32825 ONC/RPC call timed out";

The filter timeout value is set very low.

Verify that the CAI plugin registry has the correct settings. Additionally, increase the Remedy AR System server timeout settings.

The following file does not open:

//servername/D$/Workspace/UDM/
DJB000000003317/cicmdbfile.txt

The server is not in the correct domain or the port number is incorrect.

If the file is in a server group environment, verify if the servers are communicating with each other and if the file has a read and write access to the UDM path set during the configuration.

For more information, see the Knowledge Base article number 000097446.

The following error message is displayed:

12116: You do not have access
to modify the Company information
supplied. You must either have the Company added to your Access Restrictions, or have Unrestricted Access set to Yes in your People profile.

The kettle.properties file includes the AR_USER parameter, which is used and referenced from the Pentaho and UDM plugin.Comment the AR_USER parameter in the kettle.properties file. Then, restart the Atrium Integrator.

Additional information

Verify the following additional settings.

SettingsDescription
Configuring the UDM by following the recommendations
and enabling the Data Management preferences for the dataload jobs.

For more information, see the following topics:

Configuring the load file path for the BMC Remedy AR System
attachments and spreadsheets repository.

For more information, see Configuring the load file path for the BMC Remedy AR System attachments and spreadsheets repository.

CAI Plug-in Registry settings
  • Set the total threads value:
    • 3 - 6 for normal production server data loads
    • 8 or 12 for bulk data loads (if you are onboarding as an example)
  • Set the following private queue values:
    • 390620 - 390625
    • 390627 - 390629

(Click the image to expand it.)

Confirm that the forms have the correct server name.
  • UDM:ExecutionInstance
  • UDM:PermissionInfo
  • UDM:Config
  • UDM:RAppPassword

Gathering logs

Review the following log files for load error details:

Log file nameLocation
arcarte.logC:\Program Files\BMC Software\ARSystem\Arserver\Db directory.
arjavaplugin.logC:\Program Files\BMC Software\ARSystem\Arserver\Db directory.
Pentaho pluginFor information about the location, see Troubleshooting Pentaho plug-in issues.


Was this page helpful? Yes No Submitting... Thank you

Comments