Console heartbeat

You can use the console heartbeat suite of rules and scripts to monitor consoles and perform an action when a console is not responding. 

You run these functions from a consoles.txt file, which you must upload to the Automation Server.

Related topics

Administering

Console-Automation

You must add the conHbMessager rule to each console session. When triggered, it sets the console status to ACTIVE. The conHbTimerFunc function runs every 30 seconds, examining the status of all the sessions. If a console has not received a message in the previous 30 seconds, a command is sent to the console. After another 30 seconds, the console is checked again. If it still has not received a message, then the console is marked as NOT_RESPONDING, and a priority message is created and sent to all viewers.

conHbStartup

//
// Command: conHbStartup
//
// Description:
//    This command starts the console heartbeat process. It should be
//    called from the session startup command.
//
//
// The conHbLoadConsoleSession reads a text file from the Server's runtime
// directory that has the format:
//
//    sessionName,query active command
//
// The timer fires every 30 second. If a message hasn't been received
// (see conHbMessage) a "query active command" is sent to the console.
// The "D T" command is sufficient to get a response from the console.
// When the timer fires after that, it will look to see if a message
// has been received.
//
// The console heartbeat rules are provide as an example of what
// can be done. CUSTOMER REVIEW AND MODIFICATION IS NECESSARY
// BEFORE USE.
//  
//
function conHbStartup()



session messageSource



session ConsoleHbTimerValue
session ConsoleHeartbeatTimer



trace("conHbStartup: running on session " & messageSource.NAME, GREEN)



// Get the session names loaded into the array
conHbLoadConsoleSessions()



//
// ConsoleHBTimerValue is the millinumber of seconds for the ConHb command to run
//
ConsoleHbTimerValue = 30000     // 30 seconds



//
// Start up a repeating timer to watch for sessions with no messages
//
ConsoleHeartbeatTimer = createRepeatingTimer(ConsoleHbTimerValue, "conHbTimerFunc")



trace("ConsoleHeartbeatTimer = " & ConsoleHeartbeatTimer, YELLOW)



return 0

conHbMessage

//
// Filter: conHbMessage
//
// Description:
//    This filter catches all messages and sets the console state to "ACTIVE".
//    This tells the conHb timer command that the console is alive.
//
// See conHbStartup for an overviewer of the heartbeat process.
//
// Copyright (c) 2011-2012, BMC Software, Inc.
//
trigger on true



session messageSource



conHbSetSessionState(messageSource.NAME, "ACTIVE")

conHbLoadConsoleSessions

//
// Command: conHbLoadConsoleSessions
//
// Description:
//    Loads the data from sessions.txt into the hbSessions array.
//
//    See conHbStartup for an overviewer of the heartbeat process.
//
// Copyright (c) 2011-2012, BMC Software, Inc.
//
function conHbLoadConsoleSessions()

global hbSessions

if hbSessions != false then
    idx = 0
   
    do while idx < hbSessions.size()
        if hbSessions[idx].MSG != false then
            clearMessagePermanent(hbSessions[idx].MSG)
        endif
        idx = idx + 1
    end
   
endif

hbSessions.clear()

consessNames = load("consoles.txt")
if consessNames != false then
    idx = 0
    do while idx < consessNames.size()
        tokens = strSplit(consessNames[idx], ",")
        conHbAddSession(tokens[0], tokens[1])
        idx = idx + 1
    end
else
    trace("Could not load console session list", RED)
endif



return true

conHbAddSession

//
// Command: conHbAddSession
//
// Description:
//    Adds a console to the hbSessions array.
//
//    See conHbStartup for an overviewer of the heartbeat process.
//
//
function conHbAddSession(name, command)

global hbSessions

sess.NAME = name
sess.STATE = "INACTIVE"
sess.COMMAND = command


trace("conHbAddSession: " & sess.NAME & ", " & sess.COMMAND, WHITE)
hbSessions.add(sess)

return true

conHbStop

//
// Command: conHbStop
//
// Description:
//    Kills the heartbeat timer.
//
//    See conHbStartup for an overviewer of the heartbeat process.
//
// Copyright (c) 2011-2012, BMC Software, Inc.
//
function conHbStop()

session ConsoleHeartbeatTimer

if ConsoleHeartbeatTimer != false then
    writeToOperator("ConsoleHeartbeatTimer = " & ConsoleHeartbeatTimer, YELLOW)
    cancelTimer(ConsoleHeartbeatTimer)
    ConsoleHeartbeatTimer = false
else    
    writeToOperator("ConsoleHeartbeatTimer not started on this session.", RED)
endif

return 0

conHbExamineSession

//
// Command: conHbExamineSession
//
// Description:
//    Examines the state of the session and takes appropriate
//    action.
//
//    See conHbStartup for an overviewer of the heartbeat process.
//
// Copyright (c) 2011-2012, BMC Software, Inc.
//
function conHbExamineSession(conHbSession)

trace("conHbExamineSession:    state " & conHbSession.STATE, RED)
select     
    when conHbSession.STATE == "ACTIVE" then
        if conHbSession.MSG != false then
            conHbSession.MSG.TEXT = conHbSession.NAME & " Console now active"    
            conHbSession.MSG.COLOR = GREEN    
            createTimer(5000, "clearMessagePermanent", conHbSession.MSG)
            conHbSession.MSG = false
        endif
        conHbSession.STATE = "NO_MESSAGE_RECEIVED"
       
    when conHbSession.STATE == "NO_MESSAGE_RECEIVED" then
        trace("conHbExamineSession:    Sending command " & conHbSession.COMMAND, RED)
        sendCommandToSession(conHbSession.NAME, conHbSession.COMMAND)
       
        if conHbSession.MSG != false then
            clearMessagePermanent(conHbSession.MSG)
            conHbSession.MSG = false
        endif
        conHbSession.STATE = "COMMAND_SENT"
   
    when conHbSession.STATE == "INACTIVE" then
        trace("conHbExamineSession:    Sending command " & conHbSession.COMMAND, RED)
        sendCommandToSession(conHbSession.NAME, conHbSession.COMMAND)
       
        if conHbSession.MSG != false then
            clearMessagePermanent(conHbSession.MSG)
            conHbSession.MSG = false
        endif
        conHbSession.STATE = "COMMAND_SENT"
   
    when conHbSession.STATE == "COMMAND_SENT" then    
        conHbSession.MSG = createMessage(conHbSession.NAME & " Console not responding" )
        conHbSession.MSG.COLOR = YELLOW
        markMessagePermanent(conHbSession.MSG)
        sendMessageToAllViewers(conHbSession.MSG)
        conHbSession.STATE = "NOT_RESPONDING"
               
    when conHbSession.STATE == "NOT_RESPONDING" then    
        conHbSession.MSG.TEXT = conHbSession.NAME & " Console still not responding"    
        conHbSession.MSG.COLOR = RED    
end

return true

conHbListSessions

//
// Command: conHbListSessions
//
// Description:
//    Utility function to list the sessions. Useful for debugging.
//
//    See conHbStartup for an overviewer of the heartbeat process.
//
// Copyright (c) 2011-2012, BMC Software, Inc.
//
function conHbListSessions()

global hbSessions

idx = 0

do while idx < hbSessions.size()
    writeToOperator(hbSessions[idx].NAME & " - " & hbSessions[idx].STATE & ", GREEN)
    idx = idx + 1
end

return true

conHbSetSessionState

//
// Command: conHbSetSessionState
//
// Description:
//    Sets the session state.
//
//    See conHbStartup for an overviewer of the heartbeat process.
//
// Copyright (c) 2011-2012, BMC Software, Inc.
//
function conHbSetSessionState(sessionName, newstate)

global hbSessions

trace("Setting  session info for '" & sessionName & "' to '" & newstate & "'" , GREEN)

found = false
idx = 0
do while found == false && idx < hbSessions.size()
    if  hbSessions[idx].NAME == sessionName then
        hbSessions[idx].STATE = newstate
        found = true
    endif
    idx = idx + 1
end

return true

conHbTimerFunc

//
// Filter: conHbTimerFunc
//
// Description:
//    This function gets fired from a repeating timer. conHbTimerFunc scans
//    the list of watched consoles and calls conHbExamineSession for each one.
//
//    See conHbStartup for an overviewer of the heartbeat process.
//
// Copyright (c) 2011-2012, BMC Software, Inc.
//
function conHbTimerFunc()

global hbSessions

trace("conHbTimerFunc", GREEN)

if hbSessions != false then
    idx = 0
    do while idx < hbSessions.size()
        trace("conHbTimerFunc: examining session " & hbSessions[idx].NAME, GREEN)
        conHbExamineSession(hbSessions[idx])
        idx = idx + 1
    end
endif

return true

 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*