Grouping Watchpoints

This topic contains recommendations for how to create and classify Watchpoints. It is intended as a best practices guide.

The types of Watchpoints that can be created are grouped into the following classes:

Because Watchpoints are grouped alphabetically on the Watchpoint Summary page, it is useful to prefix a letter to the Watchpoint name that corresponds to the class. For example, C-Client X refers to a Client Watchpoint class that monitors a specific client. These letters help sort the Watchpoints by the category of traffic that they are watching.

Summary table of Watchpoint classes

Watchpoint class	Type	Log/MIB	Purpose	Sort by
Application	Page	Off / On	Track errors or performance by application group	Application name
Infrastructure	Object	Off / On	Track errors or performance by servers or tier	Server or tier
Key functions and transactions	Page	Off / On	Track errors or performance by function	URI stem, URI query, POST Data
Performance – Errors	Object	On / Off	Capture and log errors	not applicable
Performance – Slow Pages	Page	On / Off	Capture and log slow pages	not applicable
Performance – Dynamic Objects	Object	Off / On	Track performance of all host processing heavy request	Application name or group
Performance – Static Objects	Object	Off / On	Track performance of all static content	Application name or group
Performance – Network Problems	Object	On / On	Capture and log requests with network problems	not applicable
Client	Page	Off / On	Track errors/performance by client	not applicable

Infrastructure Watchpoints

Watchpoints categorized as Infrastructure report on the platforms and devices that make up the web application. These Watchpoints are useful for benchmarking and monitoring the servers and tiers of the web application.

Server—Watchpoints that capture traffic based on individual servers are useful for comparing performance, load, and availability across servers. Typically, logging is not turned on for these Watchpoints. These Watchpoints can be created based on a cookie value, an IP address, or a host name that identifies a specific server.
For example: (server_ip = 10.1.1.1).

Tier—Web applications are aggregations of different functional components. To complete complex end-user requests, the web servers forward portions of the work out to back-end services. Watchpoints can be created to isolate traffic based on those application tiers, but it requires in-depth knowledge of the application. Some examples of tiers by which that traffic can be grouped are web, application, reverse proxy cache, database, authentication server, and report server.
For example, (host is "rpc.mysite.com") and (host is "image.mysite,com) are examples of Watchpoints grouped by report server and authentication server respectively.

Performance Watchpoints

Watchpoints categorized as Performance consist of Watchpoints for errors, slow pages, requests with network problems, static objects, and dynamic objects.

Errors—Watchpoints can be created to captured errored requests only. Individual Watchpoints can be created for different kinds of errors or errors can be grouped into one Watchpoint. It is common practice to have a Watchpoint for application errors and one for all errors. Enable logging for Watchpoints that capture errors to see the error types, who experienced them, and what caused them.

Examples of Performance Watchpoints

Error type	Status code	Completion code	Notes
All Errors	>=400	4 or 5	This error does not include HTTP Status 401 responses.
Content not found or Forbidden	404/403	4 (error)	Watchpoint filter could be configured to check for internal referrer field. Example: ((http_status = 404) or (http_status = 403) and (referrer contains “mysite.com”)
Application Errors	500/503	4 (error)	Example: (http_status = 500) or (http_status = 503)
Frustrated Aborts	504 ¹	2 (aborts)	A user aborts a request after waiting less than 8 seconds for the host to respond. Example: (http_status = 504) and (completion = 2)
Server Timeout	504 ¹	5	A user aborts a request after waiting more than 8 seconds for the host or host does not respond in under 30 seconds. Example: (http_status = 504) and (completion = 5)

^{1 Status codes of 504 with a completion code of 5 or 2 represent requests that had no HTTP response generated by the server. The 504 code is created when the server does not respond or the response exceeds the wait time.}

Slow Pages—Watchpoints can be created to capture requests that took a long time to fulfill. You can set a value between the 99th and 95th percentile, which will log between 1% and 5% of your traffic on slow pages. Enable logging for Watchpoints that capture requests of slow pages to see which pages were slow and whether the cause was network or host latency.
For example, (e2e_time > 5000) returns pages where the end-to-end time was greater than 5 seconds.

Dynamic Objects—Watchpoints can be created to capture requests for dynamic objects. Dynamic content has to be created at the time of each request. As such, requests for dynamic content generally take longer and will include varying payload when the responses are completed. These characteristics make them more challenging to classify. Dynamic objects are usually identified with a standard file extension like .asp, .php, .jsp and .do. The host time for dynamic objects is a key metric to track for understanding the capacity and performance of web applications.
For example: (uri_stem endswith “.cgi”) or (uri_stem endswith “.asp”) or (uri_stem endswith “.jsp”).

Static Objects—Watchpoints can be created to capture requests for static objects like .gif, .jpg and .css. A Watchpoint for static objects is useful for understanding the web applications ability to handle TCP connections and deliver static content across the Internet. The host time for all static objects should be low, such as below 200ms.
For example: (uri_stem endswith “.jpg”) or (uri_stem endswith “.gif”) or (uri_stem endswith “.css”).

High Network latency—Watchpoints can be created to capture requests with high network transfer times. Enable logging for Watchpoints that capture request with high network latency. You can review the logs to identify which users are having network issues when they access the web application.
For example: (network_time > 4000).

High Host latency—Watchpoints can be created to capture requests with high host processing time. Enable logging for Watchpoints that capture requests for high host latency. You can review the logs to identify which URLs are problematic and what parameters the user was passing to them.
For example: (host_time > 3000).

High TCP RTT—Underlying a client’s web experience is the fundamental speed of the network connection on which their requests travel. Watchpoints can be created to capture requests with high TCP round trip time (RTT). TCP RTT is the equivalent of the amount of ping time from the web server to the client computer. Enable logging for Watchpoints that capture requests with high TCP RTT. You can review the logs to identify which clients are experiencing long TCP RTT or possible routing or network issues.
For example: (tcp_rtt > 300).

High TCP Retransmits / High OOO—Watchpoints can be created to capture requests with high TCP retransmits and “out of order” (OOO) segments. Out-Of-Order TCP segments are a good indicator of routing issues or poor connection quality. When segments are received out-of-order, the TCP stack must wait for the completion of preceding segments before passing the data up to the application layer. This source of network latency is most often coupled with a second indicator of line quality, high TCP retransmits. Abnormally high TCP retransmits can also indicate improperly set network parameters on either the client or the server. Enable logging for Watchpoints that capture requests with high OOO segments. You can review the logs for client IPs to identify which clients are having routing issues or packet loss.
For example: (tcp_retrans > 5) or (tcp_ooo > 5).

Key Function and Transaction Watchpoints

Within a web application, one or more transaction sequences that complete a task, are of special interest to the business. Watchpoints categorized as Key Functions consist of Watchpoints for pages like the logon page, checkout page, or account activation page. The key functions that follow a sequence, such as log on, add to cart, view shopping cart, and check out, can be prefixed with an Fx-, where x equals the step in the transaction. If the function is not part of a sequence, you can prefix the function name with F-. Under this category, Watchpoints are common for problematic pages, most requested pages, business critical pages, or pages with recent changes in application logic.

Examples of Watchpoints related to key functions and transactions

Watchpoint name	Filter
F – Activation Page	(uri_stem endswith “/account/activate.jsp”)
F – Problematic Page	(uri_stem endswith “/webservicecall.jsp”)
F1 – Login	(uri_stem endswith “/page. jsp” and uri_query(“view”) = “login”)
F2 – Add to cart	(uri_stem endswith “/page.jsp” and uri_query(“action”) = “add_to_cart”)
F3 – Checkout	(uri_stem endswith “/page.jsp” and uri_query(“action”) = “checkout”)

Page definition—An application can use several different ways to invoke a call to a page. It is common to have a unique page identifier in the URI stem or in a URI parameter that can be passed through the URI query, or in a POST parameter where the value is contained in the POST submission.

Examples:

URI stem – www.mysite.com/home.jsp

URI parameter – www.mysite.com/page.jsp?view=home

POST parameter – The HTTP method is POST and the parameters that invoke the page are passed in the POST data

Client Watchpoints

Client Watchpoints isolate traffic from a particular client group. IP address blocks, cookie values, host names, POST parameters or URI query parameters can identify clients. Customer-specific Watchpoints can be created to highlight clients with the highest internal visibility, new clients, clients that are at risk, or clients experiencing performance problems.

Examples of Client Watchpoints

Watchpoint name	Filter
C – Client X	(hostname = “clientx.mysite.com”)
C – Customer B	(uri_query(“client_ID”) = ”CustomerB”)

Page tree

Grouping Watchpoints

Infrastructure Watchpoints

Performance Watchpoints

Key Function and Transaction Watchpoints

Client Watchpoints