This documentation supports the 18.05 version of Remedy with Smart IT.

To view the latest version, select the version from the Product version menu.

Generating and interpreting KCS reports


After ensuring that necessary permissions and settings are configured, Knowledge-Centered Support (KCS) Coaches can generate out-of-the-box KCS reports. In Smart IT, when you select an option to launch a KCS report, the report opens in Smart Reporting. As a KCS Coach, you must apply the filters to get the result you want in KCS reports. The following filters are common to most KCS reports:

  • User, Article Author, Department, Company, Support organization, or Support group filters: Use these filters to review the performance of your team.
     An author creates or contributes to the article, whereas a user uses knowledge articles to resolve incidents. 
  • Article Title, Article ID and Article Status filters: Use these filters as the base for generating reports.
  • Date: Use these filters to generate reports for a specific day, week, month, or year (all displayed in digits), and also to specify time in some reports.

Filters that are used to generate KCS reports have the following features:

  • By default, without applying any filters, the report displays the results for the past one month (the date of generating the report is considered as the last date) and includes articles written by your team members.
  • When you apply filters and run the report, the system remembers the filters you applied and populates them during the next session when you generate that report.
  • Filters contain only those values for which data is available. For example, out of 100 users, only 90 users have fixed articles; the User filter on the KCS : Fix Count Report lists only 90 users. The remaining 10 users are not listed.

Notes

  • Articles created using the Decision Tree template and external file systems such as Word or PDF document are considered while generating KCS reports. You can see their details on the report however, you cannot view them in Smart IT. You can view them only through BMC Remedy Mid Tier. 
  • You must not edit out-of-the-box KCS reports. However, you can create their copy in Smart Reporting and modify them. You can also generate KCS custom reports using your preferred filters. To do so, you must have Report Writer permissions in Smart Reporting, and save them in the category created for saving KCS custom reports.
  • To display the data in reports, all reports, except the KCS : Knowledge Author Performance report, consider the relation between knowledge articles and incident tickets. The KCS : Knowledge Author Performance report consists of the citation count of articles in relation to incidents, work order, known error, problem investigation, service request, and task.
  • KCS reports are not available in Chinese, Russian, and Korean languages.
  • When the KCS Coach of a KCS knowledge author changes, the new KCS Coach can create and view the assessment results for the KCS knowledge author, using the KCS : AQI Results for Articles report. The new KCS Coach can also view the results of old assessments conducted by the previous KCS Coach, using the same report. The previous KCS Coaches continue to view the results of assessments they conducted, but cannot view the results of assessments conducted by the new KCS Coach. For more information, see KCS forms and escalations in version 9.1.01.
    Additionally, a BMC Remedy AR System administrator or an unrestricted user having any KCS permission can view results of assessments conducted by KCS Coaches for any Company.

Generating KCS reports

To generate out-of-the-box KCS reports, follow these steps:

  1. In Smart IT, click Knowledge.
  2. To generate a specific report, select its corresponding Open list.
  3. To view the report in the same or new window, select Open in this window or Open in new window.
    You are automatically redirected to the specified KCS report in Smart Reporting. 
  4. In Smart Reporting, specify necessary filters.
  5. To generate the report, click .

 

Out-of-the-box KCS reports

 KCS : AQI Results for Articles

This report provides the Article Quality Index (AQI) result of articles that you assessed. The result is used in the KCS : Article Quality Scores Report to derive the overall article quality score for a user.

Note

If you assess an article multiple times, the report shows multiple entries of each question for that article. To view the result of only one assessment, you must use the Date filter. However, if you assess an article multiple times in a day, the report will not display separate result of each assessment.

To generate this report, you can apply the following filters:

  • Company
  • Article Author
  • Article
  • Date

Example: To view the AQI result of articles written by Allen Allbrook that you assessed in the previous month, you apply the Article Author and Date filters, and generate the report.

The report displays details of Allen's articles, assessment questions, desirable answer of each question, and the actual answer that you marked for each question. You can view articles to know their nature and content. To know the weighting value of each assessment question, see the AQI Questions table displayed below the KCS : AQI Results for Articles report.

The KCS : AQI Result for Articles report helps you know how each article has performed in the AQI assessment, and find out areas of improvement for knowledge authors. You must mentor them to improve and if needed, arrange for training.

For more information on how to create AQI question sets, see Creating question sets for assessing knowledge articles in Related topics.

 KCS : Article Quality Scores Report

This report is based on the Article Quality Index (AQI) result of articles, and helps you know the article quality performance of your team members. To generate this report, you can apply the following filters:

  • Company
  • Department
  • Article Author
  • Date

Example: To view the overall AQI performance of articles written by Allen Allbrook that you assessed in the previous month, you apply the Article Author and Date filters, and generate the report.

The first part of the report displays the score recorded for issues in articles for each assessment question. Each question in the assessment has a desirable answer, which is marked as either Yes or No. If the actual answer does not match the desirable answer, a score is recorded; for example:
Question: Language Clear?
Desirable answer: Yes
Actual answer: No
Score for the Language Clear? question: 1

Out of all the assessed articles written by Allen, you assessed that language was not clear in one article. Similarly, scores are recorded for issues found in Allen's articles for all questions. Suppose you assessed five articles authored by Allen, and found 7 issues.

The second part of the report shows the overall score calculation. Allen's overall score is 74.50 %

 Overall AQI score calculation
ColumnDescriptionCalculation
Total issues foundCount of all undesirable answers for all assessed articles.0+0+1+2+2+1+1 = 7
Weighted issue count

Summation of scores marked for each question multiplied by relative question weighting for each question.

Relative Question Weighting is displayed in the AQI Questions table in the KCS : AQI Results for Articles report. You can preset these values at the time of adding question set and questions. For more information about setting AQI questions, see Creating question sets for assessing knowledge articles.

Consider the following values for Relative Question Weighting:

QuestionWeightDesirable answer
Incomplete Environment information?5.0No
Is Duplicate1.0No
Language clear?1.0Yes
Metadata correct?1.0Yes
Not understandable?1.0No
Statements incomplete?1.0No
Title not relevant?1.0No
(0*5.0) + (0*1.0) + (1*1.0) + (2*1.0) + (2*1.0) + (1*1.0) + (1*1.0) = 7.00
Max possible weighted issue countThe maximum score an author can get, if all actual answers did not match the desirable answers for all assessed articles.This score is calculated as the sum of all relative question weighting values multiplied by the total number of articles.

Total articles: 5

Sum of all relative question weighting values: 11

Max possible weighted issue count: 11 * 5 = 55

Weighted error adjustment

Weighted issue count multiplied by weighting factor for each error.

Note: The weighting factor for each error is the overall weighting factor value, which is maintained outside the question set. You or a user with Knowledge Admin permission can set this value through the Application Configuration in Knowledge Management configuration, for each company. For more information, see Weighting Factor for Each Error in the BMC Knowledge Management online documentation.

Weighted issue count: 7

Weighting factor for each error: 2

Weighted error adjustment: 7 * 2 = 14

Overall scoreThe overall article quality assessment score calculated as a percentage.

Weighted error adjustment: 14

Max possible weighted issue count: 55

Overall score = (1 - (14 / 55) ) * 100 = 74.5

This report helps you know the following:

  • Get a clear view of authors' performance, areas where they performed better, and areas that need improvement.
  • Identify specific training, and mentor authors to improve.
  • Identify top performers, mentor them to perform better and go to the next level of KCS role.
  • Find out how all authors of a department have performed; if all authors face a common issue, you can arrange for training to overcome the issue.
 KCS : Article Reuse Report

This report displays the number of times articles are reused in each status in the entire article lifecycle. When you relate an article to an incident, and use it to resolve that incident ticket, the reuse count of the article increments. The reuse count indicates usefulness of the article to resolve incident tickets. However, if you relate or pin the article to the incident without using it to resolve the incident, the reuse count of the article does not increment.

The report helps you understand the usage pattern, and the overall performance of articles when they move from one status to another. To generate this report, you can apply the following filters:

  • Company
  • Department
  • Article Author
  • Article Title
  • Article Status

Example: To find out the reuse count of articles written by your team members who belong to the IT Services department, select IT Services in the Department filter, and generate the report. Each color block indicates the status of articles, their reuse count in that status, and the reuse count range.

You can notice that most articles have a reuse count between 0-1 indicating that articles in this range were not useful to resolve incidents. Few articles have a reuse count between 6-10, suggesting only those articles are most useful to resolve incidents.

To find out articles in the Work In Progress status which are reused less than once, click the green block in the use count range of 0-1.

The Article Reuse Count report displays all articles in the Work In Progress status, article details, and their reuse count. You can view articles to know their nature and content.

This report helps you understand the following:

  • The usage pattern of articles
  • The most and least reused articles
  • The usefulness of articles in different statuses
  • Find out reasons why some articles are less reused and plan improvements
  • Find out the performance of authors, articles, departments, and company
 KCS : Citation Report

The citation count of authors is the number of times their articles are attached to the incident tickets by other users. Note that articles cited by its author are not considered in this report. This report displays the contribution of each article to its author's overall citation count. To generate this report, you can apply the following filters:

  • Company
  • Department
  • Article Author
  • Date

Example: To know the citation count of Allen Allbrook, specify his name in the Article Author filter, specify the date range to see the number of times his articles were cited in the previous month, and generate the report.

To view articles and the number of times each article is cited, click the total citation count for Allen.

To see incidents resolved using an article, click its contribution count value in the Article Fixes report.

The citation count report helps you know the following:

  • Usefulness of articles to resolve incidents
  • Top performers whose articles are mostly cited
  • Analyze why some authors are not cited, and mentor them to improve their performance
 KCS : Feedback Response Time Report

This report helps you know the timeliness of users to fix issues in articles flagged for improvement. Feedback response time is the time elapsed between flagging and fixing articles, and is an important factor in measuring users' performance in fixing issues. An article is fixed when a user selects the I'm unflagging this check box in the flagged comment area in Smart IT. The user may or may not update the article before fixing it. 

The report is generated on the basis of business hours, and displays the average response time spent to fix articles. The business time considered in this report depends on the author's support group and the business time segment defined for that support group. For information about setting business hours and holidays, see Adding business hours and holidays in the Related topics.

Consider the following example:
An office has working hours from 9:00 AM to 5:00 PM, for five days a week. Feedback 1 is received on a Friday evening; it is fixed on the following Monday morning. Feedback 2 is received on a Monday morning; it is fixed on the same day, in the evening. In this case, response provided for Feedback 1 is considered faster than that for Feedback 2.

Notes

  • The value 00:00 for an article denotes that the time elapsed between flagging and fixing is less than a minute. 
  • This report does not show the data for users whose department is blank in the CTM:People form in BMC Remedy IT Service Management.

To generate this report, you can apply the following filters:

  • Department
  • Article Author
  • Date

Example: To find out the response time to flagged articles of by your team members who belong to the Quality Assurance and Customer Service departments, you select Quality Assurance and Customer Service in the Department filter, and generate the report for the previous month.

To review the response time of the Quality Assurance department, select its average response time.

In the Article Feedback report, you can view articles, feedback given to the article, and the time taken to fix each article. You can identify response trends, and the performance of each article. You can investigate why certain issues took more time to fix, and identify areas for improvement. For more information, see "Managing flagged knowledge articles" in Related topics.

 KCS : Fix Count Report

This report helps you find out the number of times your team members have fixed issues in articles flagged for improvement. An article is fixed when a user selects the I'm unflagging this check box in the flagged comment area in Smart IT. The user may or may not update the article before unflagging it. To generate this report, you can apply the following filters:

  • Department
  • Support Group
  • User
  • Date

Example: To find out the fix count of your team members, select their name, say, Allen, Adams, and paresh in the User filter, specify the date range of two months, and generate this report.


Based on the result, you can easily compare the fix count of two months. You can analyze if the fix count rate has increased, decreased, or remained stable compared to the previous month. You can also analyze who performed better, and actively participated to fix issues in the flagged articles.

To view articles contributing to the fix count total for Allen, click the fix count for November.

The Fix Count: Article Fixes report displays article details, and the number of times Allen fixed issues in each article. Note that multiple users can fix articles however, this report displays the fix count by the selected user for the specified month. For example, an article was fixed ten times; both Allen and Adams fixed it five times each. The Fix Count: Article Fixes report for Allen and Adams will list the same article with the fix count of five each.

You can view articles that were fixed multiple times, and take measures to improve their quality. For more information, see "Managing flagged knowledge articles" in Related topics.

 KCS : Knowledge Author Performance Report

This radar chart report enables you to analyze the performance of your team members based on the following performance metrics:

  • Participation Rate - proportion of the number of times incident are closed by using knowledge articles to the total number of incidents closed.
  • AQI - overall Article Quality Index score of author in the assessment period.
  • Average Incident Handled - proportion of the number of incidents resolved to the total number of incidents assigned to a user.
  • Citation count - the number of times author's articles are attached to the incidents by other users.

For calculating the AQI, Participation Rate, and Citation Count, the current day's data is added to the previous day's data. The Average Incidents Handled value is calculated from the first day of the current month to the current day, and the average value is calculated. Except the citation count, all metrics are measured in percentage.

Before you generate this report, you must configure goal for each metric. To generate this report, you must select minimum three metrics. You can generate this report only for the period of one month. If you want to know the performance for more than one month, the radar chart is based on the cumulative data.

Note

The KCS : Knowledge Author Performance report consists of the citation count of articles in relation with incidents, work order, known error, problem investigation, service request, and task. Whereas, the KCS : Citation Report consists of the citation count of articles in relation with incidents tickets only. Therefore, discrepancies are seen in the citation count for an author on the KCS : Knowledge Author Performance report and the KCS : Citation Report. 

To generate this report, you must apply the following filters:

  • Company
  • Article Author
  • Metrics
  • Date

Example: To find out the performance of Allen Allbrook, you specify his name and company he belongs to in the Article Author and Company filter. You also select all metrics, specify the previous month, and generate the report. The report indicates that Allen has performed better and exceeded all goals.


Based on the result, you can identify Allen's strengths and weaknesses. You can plan actions such as specific training requirements to improve the performance level, or you can identify Allen as a potential KCS Coach.

 KCS : Knowledge Created from Incidents Report

This report displays details of articles created directly from incident tickets. To generate this report, you can apply the following filters:

  • Support Company
  • Support Organization
  • Support Group
  • Article Author
  • Date

Example: To know the number of articles Allen created directly from incident tickets in the past one month, in the Article Author filter, select Allen Allbrook, specify the date and time, and generate the report. When working on incident tickets, Allen created articles based on the issues that were detected for the first time, and hence they were not documented in the knowledge base.

The report helps you find out the proactivness of the service desk agents to create articles, and help other service desk agents resolve similar incidents.

 KCS : Knowledge Participation Report

This report displays the participation rate of users such as the service desk agents in resolving incident tickets using knowledge articles. The participation rate is calculated as the proportion of the number of times incident tickets are closed by using knowledge articles to the total number of incidents closed. Service desk agents are expected and encouraged to use articles to resolve incident tickets, and link those articles to the ticket. Note that only those incidents that are closed by using knowledge articles are considered in this report. To generate this report, you can apply the following filters:

  • Support Company
  • Support Organization
  • Support Group
  • User
  • Date

Example: In the Support Group filter, specify Backoffice Support and Service Desk to review the participation rate of members of these support groups over the past one month.

The higher percentage of participation indicates that the user always attaches useful articles to incident tickets, thus assisting other service desk agents to quickly resolve incidents by referring those articles.

On the KCS : Knowledge Participation Report, select the participation percentage for a user to view articles and incidents that contributed to the participation rate of the user.

On the Associated Tickets report, you can view articles and their associated incident tickets to understand the nature of issues, resolution, and article content. You may notice that multiple articles are used to resolve a single incident.

 KCS : Knowledge Status Report

This report displays the number of days articles have remained in each status. Time spent by an article in its current status is captured daily, so it gives a clear view of how speedily a department or article author is working. You can find out in which status articles remained for a longer or shorter duration. In this report, days are counted as follows:

  • A day is represented as a whole number. One day is counted as 24 hours, including weekends.
  • If an article is in a particular status for less than one day, the report shows the number of days as 0.
  • The number of days is rounded to the lower whole number. If an article is in a particular status for 2.5 days, the reports shows the number of days as 2 days.

To generate this report, you can apply the following filters:

  • Company
  • Department
  • Article Author
  • Status

Example: To find out the status of articles written by Allen, select Allen Allbrook in the Article Author filter, and generate the report .


Based on the report, you can find out articles that remained in a particular status for a long time. You can further investigate the reason for delay, and plan for improvements.

 KCS : Multiple Authors Report

This report helps you know how many article authors have collaborated and contributed to create and update knowledge articles. KCS promotes knowledge sharing and team collaboration to create and maintain best quality knowledge base. Article authors can contribute in the form of creating articles, creating a new version of articles, making minor edits, and changing status of articles. To generate this report, you can apply the following filters:

  • Article Author
  • Date

Example: In the Article Author filter list, select Allen and Adison to view their contributions towards knowledge articles for the past one month, and generate the report. The result displays a list of articles to which Allen and Adison have contributed their inputs, and the number of their contributions for each article.

To know the details of their contributions, under Author Contribution click value 2 for article KBA00000022.

The All Authors report displays names of all distinct authors who contributed their inputs to improve article KBA00000022, irrespective of the authors specified in the Article Author filter. Thus, reports can have names of authors who are not specified in the Article Author filter. In this example, Adison contributed to the first version of the article. However, Allen and Adison both contributed to create and improve the next version of the same article.

This report enables you to:

  • View each author's contribution towards articles
  • Find out active contributors
  • Mentor team members to share knowledge

Related topics

Adding business hours and holidays in the BMC Remedy ITSM Suite online documentation

Creating question sets for assessing knowledge articles

Managing flagged knowledge articles

Reviewing your knowledge team's performance using KCS reports

Was this page helpful? Yes No Submitting... Thank you

Comments