Recovering from an assertion failure

When an assertion failure occurs, usually, the server does not respond or it cannot be restarted, and the dbsrv7 log files contain the following message: Assertion Failure

Assertion failures are usually, associated with hardware IO issues involving the database file and memory errors. Although the database can usually recover from an abrupt shutdown, an abrupt shutdown can sometimes cause an assertion failure.

For more information about this issue and the various approaches to fixing it, see the Sybase support website at the following URL: http://www.sybase.com/detail/1,6904,1010805,00.html

If the Sybase database encounters an assertion failure, it cannot usually recover on its own, and you must follow a manual recovery procedure. This manual procedure requires a backup of the database. If you do not have a backup, it is sometimes possible to recover the existing database, by using Sybase utilities. If this does not work, you can contact the Sybase Support, and they can sometimes recover the existing database (Sybase usually charges for this service.)

To recover from an assertion failure

If an assertion failure occurs, perform the following tasks to begin recovery:

1. Run the pw dump1 command to preserve all files for later analysis.

2. Check the files in the area designated for Infrastructure Management Database archives to ensure that any existing backups or archives of the database are secure.

3. Make a copy of the following database files located in the installationDirectory/pw/sybase folder:

   • The transaction pronto.log file.

   •  The database storm_<hostname>.db (hostname is the name of the BMC TrueSight Infrastructure Management Server computer. )

4. Run the following command to restart the database: pw system start. Sometimes, the database recovers after a second attempt at restarting.

5. Check towards the end of the installationDirectory/pw/asatmp/storm_db.log file to see if:

   • The database startup is proceeding. 
   • The database is in recovery mode. 
   • The database has encountered another assertion failure.  
   • You must allow the startup to proceed if you observe the message, database recovery in progress. This indicates that the database is trying to reconcile the transaction log file with the .db file.
   • If you see another Assertion failed message, or if the database fails to start up for any reason, notify the BMC Customer Support.

6. If the database starts up successfully, run the following command to ensure that the database is in a stable condition: pw database validate

7. Notify BMC Custom Support if the validate command returns errors.