Event Basics

The following sections provide basic information about events in PATROL.

Agent Generated Events

To understand the PEM architecture, it is important to realize that there are a number of ways to trigger an event.

After an event is triggered, the PATROL Agent feeds a snapshot of the event to the PEM. The PEM then matches the information passed on by the agent against an event catalog (by default, the Standard Event Catalog). Criteria for type of event, as well as the extraction of details, are matched against this catalog, and an event type is established.

In addition to the Standard Event Catalog, the PATROL Developer Console can be used to build custom event catalogs. The Standard Catalog describes types of events, and the PEM pushes any event from any performance area through this catalog. Custom events are sent from custom triggers, also user-definable from the Developer Console, and pushed through the PEM. The PEM then forwards all details as one event file, complete with numeric id, to the Event Repository.

The Event Repository is a definable amount of memory dedicated to buffering the most recent events noted by the PATROL Agent and PEM. PEM also forwards these event files as messages to any interested PATROL Console via the PATROL API, as well as any registered SNMP console via an SNMP trap. After the Event Repository lls to the maximum level, it writes itself to a circular file of the same size, called the Event Log.

When an event occurs, the state change is propagated up through the object hierarchy (in the case of alarms and warnings). The agent uses this means to notify you visually on the console. At the same time, the agent logs the event details, which it stores as a circular log file.

The agent sends the most recent event entries to the event cache at your console. You can dictate how much temporary disk on your console you will devote to the cache using the miscellaneous option on the User Preferences Property Sheet.

The rest of the events are stored in the log file at the agent. You query this file when you run View > Event Repository Statistics from the Console Menu. You can also query the log file at the agent using the Filter button in Filter Mode.

KM generated events

Out-of-the-box events are automatically triggered by changes in state of objects. PATROL does not issue events from recovery actions within parameters, but rather hard codes them into the product. So, with this model, you get coverage from the first time you start up PATROL. IF something is going wrong, PEM sends an event. If something comes out of ALARM (is going right), PEM sends an event. If someone attaches to an agent or runs a command remotely on an agent, PEM sends an event.

What if you want to specify what constitutes an event in your environment? You can trigger events with one simple PSL command called event trigger(). If a performance area breaches a threshold; you can use the PSL command from within a Recovery Action (which is initiated immediately upon notice of the ALARM or WARNING).

If one event occurs, and you want to issue another based on other conditions at the time of record, you can issue the PSL command through an existing event type.

If you want operators the ability to manually issue events as trouble tickets to the OEM trouble ticket system, you can use the PSL command as a PATROL Menu Command on the PATROL Console.

Event Catalogs

The PATROL Standard Event Catalog gives you the broadest event coverage possible. No matter what KM you purchase, or develop on your own, the Standard Event Catalog ensures that you get a basic set of events with the default triggers.

All KMs will have their ALARMs, WARNINGs, and other default events passed through the Standard Event Catalog. No matter what the KM, or individual performance area within that KM, certain events will always be passed along as a type from within the Standard Catalog. For example, an Event Class 11 from the Standard Catalog is de ned loosely as any change from OK to ALARM, with a corresponding Recovery Action. Any event, whether from ORACLE, the CPU area of the PATROL KM for UNIX, or a custom KM, that matches the criteria will fall under Event Class 11. This creates a small issue in providing event specific help, troubleshooting, or other documentation as part of the event generation.

Custom Event Catalog

However, where such event specific documentation is desired, a custom event can be established. Like any other attribute of a KM, events can be added individually to a pick list. This pick list then becomes a Custom Event Catalog. Each one of these custom events can then define one specific event, and document details concerning the automated response from PATROL, or the desired response from the end-user/operator. This custom event is generated upon the trigger you select, to ensure it is only created if the specific situation arises.

Elements of an Event Class Dialog

Event Type: Default type is the type of event (warning, alarm, etc.). Life expectancy: has to do with whether or not the event entry will be written into the circular log file from the memory block.

SNMP Support: allows you can select if the event is to be sent as an SNMP trap to another third-party console or trouble ticket system. Escalation Period: defines if the event is not acknowledged, closed or deleted in the time specified here, the commands you enter in this field will run. Use this only if PATROL Consoles will be the component from which you assign ownership.

Notification Command is executed as soon as this type of event is logged. Use this for your automated solution, if any.

Acknowledgement Command is triggered when an end-user physically selecting this event when logged, and designating it as "acknowledged". Again, this is dependent upon the PATROL Console being responsible for ownership and closure.

Expert Advice/Description You also have the ability to enter expert advice. This field can be passed along to other OEM systems. Description is also customizable from this Property Sheet and also passed along to other OEM systems.

The steps below demonstrate how to create a custom event catalog. In this example, let's assume that you created a parameter called CPUNumProcs in the NT CPU application class, and now you need to build an event catalog and trigger a recovery action for the CPUNumProcs parameter

1. From the KM tab tree view, expand the [Application Class ] folder.
2. Expand the [NT CPU ] application class.
3. Expand the [Global ] folder.
4. Right-click Event Catalog and select New. The Event Class property sheet appears.
5. In the Event Type field, select ALARM.
6. In the Name field, type "CPU Working Too Hard".
7. Select the [Description ] tab. In the Description area, type "I'm working too Hard!!! There are %s process running and my CPU utilization is %s."
8. Click [OK ].

State Change Actions

State change actions are console-side actions that will be taken in case an object changes state. Below you will find a list of facts about state change actions.

• State change actions can be defined on computer level and application (actually instance) level.
• State change actions are executed by the console and will only be executed when the computer/instance changes state.
• If you defined a state change on instance level, then if one parameter of your instance parameters goes into alarm (which changes the state of the instance to alarm), the alarm state change action will be red off. If a bit later another parameter will go into alarm, this will not cause the state change to be re-triggered, because the instance will not change its state again!
• If you define a state change to a computer class, only computer objects will trigger the action. Also, computer objects don't get their state from parameters, even when everything is set to propagate. They get their state directly from the parents of those parameter's instances (applications). Therefore, there's no 'direct' way for a computer class to know which parameter put it into alarm.
• State changes can be OS commands only.
• State change actions are very different from recovery actions, which are associated with parameters and event notification actions, which are associated with event. The agent executes both recovery actions and event notification actions.
• State change actions do not have an "auto repeat until..." possibility, but that doesn't mean this functionality can not be developed by you. For example: on ALARM/WARN state change, invoke some kind of OS process such as a C program, perl script, OS script, etc. which would continuously run and periodically take the desired action. You would then use the OK state change to run another OS command which would stop the ALARM state change action.

Macros in state change actions

• If you define state change actions on a global level, and additionally on a localized level (for a specific computer/instance) then both the global and the local state change action will re off in parallel.

In a state change action, following macro's can be used:

These macro's will be substituted prior to executing the OS Command.

echo %{HOSTNAME}

PatrolCli PSL execute

The PatrolCli utility is one of the programs that is almost completely written on top of the event subsystem. PatrolCli also allows you to run PSL, but there is a limitation in the size of the PSL script that it can run. However, if you understand how the agent executes notification actions, we will see that we can overcome that limitation. The symptom of the issue is that only the first line of a multiline PSL script will be executed (the rest seems to be ignored). In this section we will investigate the issue and find a solution for it.

Let's say you want to run a PatrolCli command that will return all loaded applications. Code to do that is:

applications=get vars("/","subnodes");
foreach appl (applications)
{
applpath="/".appl;
if ( get(applpath."/__type__") == "APPLICATION")
{
ret data=[ret data,applpath];
}
}
# Now we want to return ret data.

When you would trace the remote PSL process that is launched, you would see that actually all lines will execute, but you will only get the result of the first line. Which is:

applications=get vars("/","subnodes");

The reason for this is that your execpsl is actually being executed in the RemPsl standard event's notification action. The code in the RemPsl standard event notification action is basically:

# This command assumes that event arguments are separated by tab
xxx pem=%fEV ARG1g
event trigger("0","Result","INFORMATION",
"6",xxx pem,"%{EV_ARG2}");

The first line in the RemPsl standard event notification action will cause the execution of your script since a replacement of %{EV ARG1} will occur with your PSL script before compilation and execution.

This will thus result in

xxx pem=applications=get vars("/","subnodes");
foreach appl (applications)
{
applpath="/".appl;
... etc ...

This means that only the result of first line will actually be passed back through the event trigger().

A way to execute multiple lines and get the result of what you want it to be is by assigning the result to xxx pem yourself in your script. That means the last line should be:

# Now we want to return ret data ...
xxx pem=ret data;

Another (simple) example to this would be

a="hello";
xxx pem="a.world!";

Could be executed like this:

execpsl "0;\na=\"hello\";\nxxx_pem=a.\" world!\";\n"

And will result in the following script to be executed in the notification action:

xxx pem=0;
a="hello";
xxx pem=a." world!";
event trigger("0","Result","INFORMATION",
"6",xxx pem,"%{EV_ARG2}");