Troubleshooting package import issues

When a package is not imported successfully, you may not see any error message on the AR System Deployment Management Console. See the guidelines in this topic to obtain the appropriate logging and troubleshooting steps.

Symptoms

  • Unable to view the package post zip file import is finished
  • Package import fails with 'Init Error'
  • Package fails to upload the zip file, status gets stuck in importing state.

  • Package import fails with message:

    Error in plugin :ARSYS.ARF.ARMIGRATE

Scope

  • One or more users experience the problem. 
  • In a Server Group environment, this could affect one or more servers

Resolution

Perform the following steps to troubleshoot issues while importing packages:

Step

Task

Description

1

Review the Configuration

  1. Ensure that deployment packages can communicate with the ARMigrate plugin:
    1. Ensure that the ar.cfg file contains an entry for Server-Plugin-Alias: ARSYS.ARF.ARMIGRATE.
      ar-cfg.png

    2. Go to ARSystemInstallDirectory/BMC Software/ARSystem/pluginsvr/ and ensure that the pluginsvr_config.xml file contains definitions for the ARSYS.ARF.ARMIGRATE plugin and all required paths. All .jar files must be present at the defined locations.
      pluginsvr-config.png

    3. Ensure that the AR System Monitor form contains correct hostname entries for each monitor type that is a part of the environment: AR System server, Mid Tier, and Smart IT server. Delete any incorrect or duplicate host entries from this form.
      arMonitor.png
    4. If you have made any changes as per above steps, you must restart the AR Java plugin server. See How to restart the AR Java plugin server.
    5. Compare monitor ID for each monitor type between AR System monitor form and the monitor-<application>-guid.properties file. The file locations are:
      monitor-ARServer-guid.properties: ARSystem_InstallDirectory
      monitor-Midtier-guid.properties: MidTier_InstallDirectory\filedeployer
      monitor-SmartIt-guid.properties: SmartIt_InstallDirectory\filedeployer

      If the Monitor IDs do not match, perform the following steps to generate the file with new GUID and update the AR System monitor form:
      1. Delete the monitor-<application>-guid.properties file.
      2. Delete the entry from the AR System monitor form.
      3. Depending on the monitor type, restart the AR System service, MidTier file deployer service, or the Smart IT file deployer service.

  2. Check the AR plugin server heap size in the armonitor.cfg file. The minimum heap size must be 3 GB (3072m). Run the following command to specify the heap size as 3 GB:
    "%BMC_JAVA_HOME%\java" -Xmx3072m -classpath "ARSystemInstallDirectory\pluginsvr"

    For example:

    "%BMC_JAVA_HOME%\java" -Xmx3072m -classpath "C:\Program Files\BMC Software\ARSystem\pluginsvr"
  3. Open SHARE:Application_Properties form and review the version for property value:
    BMC Remedy Action Request System, BMC Atrium CMDB, BMC Atrium Integrator, BMC Remedy Asset Management, BMC Remedy Incident Management, BMC Remedy Change Management, BMC Remedy Problem Management, BMC Remedy Knowledge Management, BMC Service Request Management, and BMC Service Level Management.
  4. Review the requirements
  5. Verify the downloaded package zip file size with the original file on the server.
  6. Enable the arjavaplugin.log in DEBUG mode. The logs are generated in the ARSystemInstallDirectory/ARSystem/db directory.
    See How to enable DEBUG Java plugin server logging.
  7. Delete the existing failed package from the deployment form.
  8. Import the package again.
  9. After the package is imported, view the state and status reason fields. For a successful import, the status reason field displays Success:ARMIGRATE.
  10. If the package fails to import, continue to the next step.

2

Collect the logs

Identify and record any related errors in following logs in ARSystemInstallDirectory/ARSystem/db directory.

Enable the arjavaplugin.log in DEBUG mode before importing the package and reproduce the issue. Copy these logs to another location where you can review them so that they do not get overwritten. Note that a few logs may be large in size depending on use case performed.

3

Disable logs

Disable the arjavaplugin.log by changing the log level to INFO.

4

Create a BMC Support Case

Collect and send logs and detailed information when creating a case with BMC Support:

  1. Provide the following information as part of your case:
    • Name of the package
    • Package status and error message
    • Screenshot of error

  2. Collect following files:

    • ar.cfg
    • armonitor.cfg
    • pluginsvr_config.xml
    • arjavaplugin.log

    To know the log file locations, see AR-System-server-file-locations.

  3. Create a zip file of the collected logs and either attach the zip file to the case (up to 2 GB) or send the file to BMC by using an FTP.

Cause and resolution for error messages

Once you have determined a specific symptom or error message, use the following table to analyze the issue:


Issue

Cause and resolution

Package import or deployment failed without any specific reason on the AR System Deployment Management Console, but log may contain the OutOfMemory error.

Insufficient physical memory on the system where the plug-in server is running.

During import, the deployment plug-in extracts the information from the deployment package. However, due to less physical memory, the package extraction fails, therefore, the package is not imported. Check the arjavaplugin.log file for information about the operation that resulted in import failure. For example, if the package size is 300 MB, and if there is no free space to create the ZIP file structure after extraction, the package will not be imported.

Solution: Increase the physical memory on your system.

The following error is logged:

ARMigratePlugin Initializing the thread pool completed.

Insufficient plug-in Heap Size

Typically, the size of packages with payload content is 250 to 300 MB, which needs a higher heap size for the package import to succeed.

The AR System Deployment Management Console does not show any error message related to this. Check the arjavaplugin.log file for information about the cause of failure. If you see this error message in the log file, it means that the AR Migrate plug-in has been restarted. Always check for the timestamp for this message when debugging.

Solution:

If the heap size is less than 1024 MB, increase the heap size on all the servers in the server group. If you have multiple plug-in servers, update the heap size on the plug-in server where the AR Migrate plug-in is deployed.

To increase the heap size, open the armonitor.conf file and add -Xmx1024m value to the Java parameter, and then restart the plug-in server.

For example:

"%BMC_JAVA_HOME%\java" -Xmx1024m 
"<AR_Install_Dir>\ARSystem\pluginsvr;
<AR_Install_Dir>\ARSystem\pluginsvr
\arpluginsvr<version>.jar;
<AR_Install_Dir>\ARSystem\arserver
\api\lib\arcmnapp<version>.jar" 
com.bmc.arsys.pluginsvr.ARPluginServerMain
-x <serverName> -i 
"<AR_Install_Dir>\ARSystem" -m

If you face deployment error, as a best practice, ensure that the heap size is 3 times more than the package size.

The following error is logged:

ARMigrate:java.io.FileNotFoundException: C:\BMCData\ARSystem\ARServer\Db
\BMC_DEPLOYMENT_PACKAGES\
IDGI08B9ID0GZAP4DWE4P3GZ5RLSHG\
_validateItems.md5 
(The system cannot find the file specified)

This error might also appear when the following files are not found in the deployment package:

  • _items.zip
  • _meta.arx
  • _package_guid.txt

Package is empty.

Sometimes, due to network issues, the package is not downloaded completely. When you try to import a package that was not fully downloaded, the extraction of the ZIP file fails. If the ZIP file is not extracted correctly, the md5 checksum file and signing information is also not read and exceptions are logged.

The Deployment Management Console does not show any error message related to this.

Solution: 

  1. Ensure that the package is downloaded completely before you start importing it. You can compare the file sizes of the package before and after downloading.
  2. Check arjavaplugin.log for more details.
  3. Check the Java plugin server heap size in the armonitor.cfg (Windows) or armonitor.conf (UNIX) file. For more information, see Review the configuration task in (step 1) in Resolution.

The following error is logged for some files:

Access is denied.

No permission to access theBMC_DEPLOYMENT_PACKAGE folder.

During the package import process, the package reads some configuration files and if there is no permission to access the BMC_DEPLOYMENT_PACKAGE folder, the import fails due to the inability to read the configuration files available in that folder.

The Deployment Management Console does not show any error message related to this. Relevant error message is logged in the arjavaplugin.log file.

Solution: 

  1. Provide Read/Write permission to the BMC_DEPLOYMENT_PACKAGE folder and its parent folders.
    The BMC_DEPLOYMENT_PACKAGE folder is available at the following location:
    (Windows) \AR_Install_Dir\Arserver\Db
    (UNIX) /opt/bmc/ARSystem/db
  2. Check arjavaplugin.log for more details.

The following error is logged:

"ARMigratePlugin: ImportPackage failure to validate checksum of package!: <package_entry_ID>"

Package contents are modified.

If you modify a BMC package and import it, it results in a mismatch of the Md5 checksum information of the original package and the modified package. Import process fails due to mismatch in the Md5 checksum value.

Solution:

  • Do not modify the BMC packages.
  • Do not extract the package contents on the same location as the zip file.

For more information, see the knowledge article DOC-117791 on BMC communities.

The log file may show the OutOfMemory error and AR System Deployment Management Console may show export or import error.

Package is not uploaded completely after import.

Sometimes, it is observed that, a package is imported successfully, but the package contents are not fully uploaded to the AR System forms.

You can export the imported package from the console and compare its size with the actual package that was imported. If the size is 0, you should wait for sometime before the export operation gives the right size after all package contents are fully uploaded to the AR System forms.

If you observe that the import process taking a longer time even after refreshing, consider deleting the package from the console, and import the same package from the command line. For more information about importing a package from the command line, see Using a command-line interface to manage a packagein the BMC Remedy AR System online documentation.

AR System Deployment Management Console shows the following message:

Unable to import the package.

Mid Tier is unable to upload the package.

This issue may occur when the deployment package size is large (for example, more than 300 MB), Mid Tier cannot upload the package and is unable to communicate with the plug-in server. Due to this, the import operation is never completed. 

Solution:

  1. Ensure that the Tomcat heap size is three times the size of the deployment package.
  2. Use the command line interface only for importing the package. Use the Deployment Management console for deploying the imported package.

For more information about importing a package from the command-line, see Using a command-line interface to manage a packagein the AR System online documentation.

AR System Deployment Management Console shows the error message:

Error in plugin :ARSYS.ARF.ARMIGRATE (ARERR 8753)


<Can we add a cause here?>

Solution:

  1. Ensure that the ar.cfg (Windows) or ar.conf (UNIX) contains an entry for the ARSYS.ARF.ARMIGRATE  plugin.
  2. Check the arjavaplugin.log file for more details.

For more information, see the knowledge article DOC-109135 on BMC communities.

The deployment package is imported until 95% and then the package becomes unavailable. The mid Tier log contains the following message:

ERROR in preload, aborting preload for server <server-name>

 ARERR [91] RPC call failed : <server-name> Java heap space

Mid Tier is unable to preload the package.

This issue may occur due to insufficient heap size for the Mid Tier server.

Solution:

  1. Go to the Tomcat installation directory and run the tomcat7w.exe file.
  2. Increase the INITIAL & MAX Memory Pool size to 4 GB.
  3. Restart Tomcat.

Unable to upload a hotfix zip file by using the AR System Deployment Management Console. The log file contains the following error:

ImportPackage failure FileNotFoundException

Incorrect zip file.

This issue may occur if you specify incorrect zip file for import.

Solution:

Ensure that you specify the correct zip file for import. While importing a hotfix zip file, extract the zip file and locate the correct package zip file for import. For more information, see the knowledge article DOC-109954 on BMC communities.

Related topic

Support-information

Troubleshooting-issues-with-deployment-packages

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*