Monitoring access log

Web servers and proxy servers record client access information in an access log file. BMC PATROL for Internet Servers can process these access log files to report management information that is meaningful to you.

To monitor your web server's access and error logs, you must load the ISM_LOGMON_BASIC.kml log monitoring file.

The ISM_LOGMON_BASIC.kml file contains the following application classes, which provide basic and advanced log monitoring, such as recording the top n clients that have accessed your website:

• INET_Web_AccLog.km
• INET_Web_ErrLog.km
• INET_Web_AccLogStat.km
• INET_Web_ErrLogStat.km
• INET_Web_Url.km
• INET_Web_ClientHost.km
• INET_Web_User.km
• INET_Web_VirtualServer.km

 Specifying the access log location

This task describes how to specify the location of the access log.

To configure the access log location

1. Access the INET_Web_Server or INET_Proxy_Server application instance menu commands.
2. Select one of the following commands:
   • On Unix computers, PATROL Admin > Configure > Files
   • On Windows computers, KM Commands > PATROL Admin > Configure > Files
      The Configure dialog box is displayed for the specified server instance.

3. Verify the path name in the Access Log text field, or type the correct path name.

   Warning

   Note

   When the location of the Access Log is relative to the server root, use a relative path name for the Access Log field; otherwise, specify an absolute file path.

4. Select Accept to save the entry and close the dialog box.

Configuring the access log format

Depending on your server setup, the access log file can be written in different formats. BMC PATROL for Internet Servers can process log files in one of three formats: Common Log Format, Combined Log Format, and W3C Extended Log Format.

 Common Log format

The Common Log Format (CLF) is a standard format that almost all web servers support. It contains the following:

• Client Host—either the IP number or the DNS name for the client
• identd information for the client (largely unused)
• Authenticated User —contains the user name of the client when authentication is used on the web server
• Request Time Stamp—indicates the time a request was made
• Client Request—HTTP request sent by the client, which includes a Method and a URL
• Response Status—HTTP response status returned by the server
• Response Size—size (in bytes) of the response returned by the server

 Combined Log format

The Combined Log Format adds the Referrer and User Agent fields to the fields of the Common Log Format:

• Referrer—URL that contained the link followed for the client's requested URL
• User Agent—browser used by the client

 W3C Extended Log format

The W3C Extended Log Format is used only by Microsoft IIS Internet servers. It contains information similar to the Common and Combined Log Formats but may contain many more possible values. For Microsoft IIS, BMC PATROL for Internet Servers determines which values are being reported. BMC PATROL for Internet Servers can report specifically on the following two additional values:

• Processing Time—total amount of time the server took to process the client request
• Virtual Server—IP address for the virtual server that was the target of the client request

To configure the access log format

1. Select one of the following commands:
   • On Unix computers, PATROL Admin > Configure > Files
   • On Windows computers, KM Commands > PATROL Admin > Configure > Files
      The Configure dialog box is displayed for the specified server instance.
2. In the Access Log Format box, select one of the following supported log formats:
   • Common Log Format (CLF)
   • Combined Log Format (CLF with Referrer and User Agent)
   • W3C Extended Log Format
3. Select Accept to save the entry and close the dialog box.

Configuring the access log statistics

This task explains how to configure the access log statistics by adding, deleting, and editing access log statistics sets.

Before you begin

When initial discovery is complete, verify that application icons appear for each loaded application instance. If the icons do not appear for each specified application instance, there may be a problem with your PATROL installation. Refer to the PATROL Installation Guide for help.

To add an access log statistics set

1. Access the server instance commands.
2. Select one of the following commands:
   • On Unix computers, PATROL Admin > Configure > Files
   • On Windows computers, KM Commands > PATROL Admin > Configure > Files
      The Configure dialog box is displayed for the specified server instance.
3. Select Access Log Statistics: Edit ; then Accept to display a dialog box indicating which statistics sets have been defined and activated (Utilized Statistic Sets) and which have been defined but are inactive (Unused Statistic Sets ).
4. Select Add.
5. Select the statistics set that you want to add from the list of Unused Statistic Sets.
6. Select Accept to save the entry and close the dialog box.
    A new configuration dialog box is displayed.
7. Select a Log Event from the following menu options:
   • NONE
   • INFORMATION
   • WARNING  
   • ALARM
8. From the Select all requests which... menu, select do NOT match or match to specify whether the request should match the defined conditions.
9. Indicate the number of access log collection cycles that can occur before an event will be reported in the Console.
   • Selecting Immediately On Event causes the event to be reported immediately when the event is detected.
   • Selecting After Event Occurs N Times with a value of n causes BMC PATROL for Internet Servers to trigger an error after the event has occurred in n consecutive collection cycles
10. Build the conditions for the new statistics set in the remaining text fields. Each of these expressions is combined with a Boolean AND statement to form the test condition. These expressions can be text expressions or UNIX regular expressions.
11. Select Accept.
     The new statistics set appears along with the sets that were being used.

To delete an access log statistics set

1. Select one of the following commands:
   • On Unix computers, PATROL Admin > Configure > Files
   • On Windows computers, KM Commands > PATROL Admin > Configure > Files
      A Configure dialog box for the specified server instance is displayed.
2. Select Access Log Statistics: Edit ; then Accept to display a dialog box indicating which statistics sets have been defined and activated (Utilized Statistic Sets ) and which have been defined but are inactive (Unused Statistic Sets ).
3. Select Delete.
4. Select the statistics set you want to remove from the Utilized Statistic Sets list.
5. Select Accept to display another dialog box containing information about the statistics set that will be deleted.
6. Select Accept to confirm the deletion. The deleted statistics set appears in the Unused Statistic Sets list.
7. Select Done to close the dialog box.

To modify an access log statistics set

1. Select one of the following commands:
   • On Unix computers, PATROL Admin > Configure > Files
   • On Windows computers, KM Commands > PATROL Admin > Configure > Files
      A Configure dialog box is displayed for the specified server instance.
2. Select Access Log Statistics: Edit ; then Accept to display a dialog box indicating which statistics sets have been defined and activated (Utilized Statistic Sets) and which have been defined but are inactive (Unused Statistic Set s).
3. Select the statistics set that you want to modify and select Edit.
4. Select Accept.
    A new configuration dialog box is displayed.
5. Complete the dialog box to reflect the necessary changes.
6. Select Accept to save the entry and close the dialog box.
7. To modify additional statistics sets, repeat steps 1 through 6; otherwise, select Done to close the dialog box.

Modifying access log statistics by using PATROL Configuration Manager (PCM)

You can specify custom values for all the Utilized and Unused Statistics. You can specify the custom values by editing the default values present in the pconfig file under /InetSetup/INET_Web_Server/Defaults/. You must specify the custom values in the pconfig file before adding web servers for monitoring.

For more information on how to use the PATROL Configuration Manager or the wpconfig utility, see BMC PATROL Configuration Manager User Guide.

To specify custom values for access log statistics

1. In the pconfig file of a selected PATROL Agent, go to /InetSetup/INET_Web_Server/Defaults/.
2. Select the required statistics.
3. Modify the default values as needed.
4. Apply the configuration.

Adding, editing, or deleting access log statistics

This section describes how to add, edit, or delete access log statistics by using PCM.

To add an access log statistic by using PCM

1. Create a name for the new statistic that you want to add.

   Warning

   Note

   • The name must not be the same as the name of an existing statistic.
   • The statistic name can contain any uppercase or lowercase letters or numbers (0-9).
   • The statistic name must not contain special characters or symbols.
2. Add the new statistic for the appropriate web server.
   /InetSetup/INET_Web_AccLogStat/Web-server_name_port/ newstat.
3. Add the new statistic in the PATROL Configuration Manager (PCM) variable **, selecting *Edit.
    The PCM variable *** is located under /InetSetup/INET_Web_AccLogStat/Web-server_name_port/*.
4. In the PCM, go to the edited variable at /InetSetup/INET_Web_Server/Web-server_name_port /edited and change its value to 1.
5. Apply both the *** and edited variables to the respective agent.
6. Apply the paramSettingsStatusFlag (value 6) variable.

To edit an access log statistic by using PCM

1. Select the statistic that you want to edit:
   /InetSetup/INET_Web_AccLogStat/Web-server_name_port/ stat_name
2. Edit the statistic using PCM as follows:
   1. Go to the edited variable located at /InetSetup/INET_Web_Server/Web-server_name_port/edited and change its value to 1.
   2. Apply the edited variable in PCM.
   3. Apply paramSettingsStatusFlag (value 6) variable.

3. Apply the configuration to the required PATROL Agent.

   Warning

   Note

   You can also use PATROL Configuration Manager to edit an access log statistic.

To delete an access log statistic by using PCM

1. Delete the statistic from the PCM variable*** located at /InetSetup/INET_Web_AccLogStat/Web-server_name_port/*.
2. Go to the edited variable located at /InetSetup/INET_Web_Server//Web-server_name_port/edited and change its value to 1.

3. Apply the configuration to the respective PATROL Agent.

   Warning

   Note

   You must set the value of the edited PCM variable to 1 to ensure that the changes made to the access log statistics are reflected.

Viewing the access log reports

This section describes how to view the access log report of a web or proxy server.

1. Access the menu commands for the server instance.
2. Select the menu command View > Access Log Reports.
3. Using the View Access Log Statistics selection bar, select how many of the past days you want to examine.
4. Complete the Statistic Set, Top, and Criterion information.

5. Indicate whether you want to resolve IP addresses.

   Warning

   Note

   Do not select Lookup IP Addresses if you do not want BMC PATROL for Internet Servers to report host names in its access log reports. You may want to turn off DNS lookups in this manner to conserve system resources used in looking up host names.

6. Click Accept to view the Access Log information.
7. Analyze the following statistics that are displayed:
   • Total # of Requests. The number of requests processed by the server during a one-minute interval.
   • Request Rate (per minute). The rate at which requests were received by the server during a one-minute interval.
   • % of ALL Requests. The ratio of the number of requests for each item listed to the total number of requests.
   • Total Size of Responses (KB). The sum response size of all responses for each item listed and for all items (in kilobytes per minute).
   • % of ALL Response Size. The ratio of this page's total response size to the total size of all responses.
   • Average Size of Responses (KB). The average size of responses processed for each item listed and for all items in the list (in kilobytes per minute).