IX plan for log enrichment
Project information
Product and release | BMC Helix Log Analytics |
|---|---|
Features | Log enrichment |
Content developer | Swati Malhotra |
Epic / Use cases | Enrich logs by using external sources like DNS servers, etc. |
Design documents | |
Personas | Administrators |
Master space | |
Whatfix self-help plan | NA |
Product style sheet | Style-sheet-for-BMC-Helix-Operations-Management (Shall we follow BHOM stylesheet?) SR: If it is a separate product and space, create a separate style sheet please. Thank you. Sure. |
Use cases
Use case | Persona | Situation | Customer's information need | Delivery medium | Search keywords | Real-world example | Test case | Testing needs R&D help? | Review comments |
|---|---|---|---|---|---|---|---|---|---|
Enable Steve to enrich log entries by using external sources like DNS Servers, CSV files etc. so he can enrich logs with meaningful information that helps in faster troubleshooting. | Administrator | Happy Path | Can I add custom fields to the plain log entries to make them more meaningful? | In a Wiki topic (Enriching logs), we begin by telling users about the capability, various options available to enrich logs and then introduce the child topics. We also explain the whole process with an image. We will create the following child topics:
| log enhancement, custom tags to logs, log enrichment, adding fields to logs, DNS enrichment, LDAP enrichment, GeoIP enrichment, CSV enrichment, IP Whitelist enrichment, IP Blacklist enrichment | To get from RnD for each external source. (conversation started with dev) SR: | logs shall get enriched with the inputs from external sources or CSV file. | Yes | SR: In the use case, try to identify why Steve wants to to this. SM: Done |
What sources can I use to add such values to the log entries? | Intro on the Enriching logs topic | ||||||||
How do I use these sources? | Child topic for each enrichment source | ||||||||
Can I give a condition when these values be added to logs? | Creating an enrichment policy page | ||||||||
Can I continue to see these enhanced logs in Log Analytics dashboard? | Intro on the Enriching logs topic | ||||||||
Can I use these values in Helix Dashboard? | Intro on the Enriching logs topic | ||||||||
Can I add alerts using these values? | Intro on the Enriching logs topic | ||||||||
Can I create dashboards in Kibana by using these values? | Intro on the Enriching logs topic | ||||||||
What if I have some custom tags* like information that I want to add to logs? *Tags are not added by using any source. It will be a custom requirement of a user. | Child topic for tags as a enrichment source (I will wait for the feature to come out to decide exactly how we will cover it in docs.) | ||||||||
Corner case | |||||||||
Troubleshooting |
Wiki structure
In the outline, list the sets of tasks, concepts, and reference information that forms a complete workflow for the use case. Depending on the complexity of the feature, you might have multiple workflows or parent/child workflows. If possible, try to keep topics only three levels deep (L2 - L4). If a topic contains a help context ID, review guidelines on IDD Central before renaming the topic.
Role | L1 - Branch | L2 | L3 | L4 | L5 | Subheadings | Topic type | Rich media | Writer notes | Review comments |
|---|---|---|---|---|---|---|---|---|---|---|
All | Release notes and notices | SR: Please include the plans for the enhancement blurb. SM: Sure. I had forgotten to include it here. | ||||||||
2022 enhancements | ||||||||||
Enrich logs with external data | SR: "Enriching log messages with external data" Are they enriching the logs, log messages, log records, or log events? Enriching logs makes it sound like they are changing the log source. It that is the case, then okay. | |||||||||
Admin/Operator | Enriching logs for enhanced diagnostics | Process overview | Video | SR: "Enriching log messages [for monitoring]" SM: I will confirm it with the PM. It is to make logs more meaningful. SR: "...for enhanced diagnostics" is good, but are they enriching the logs, log messages, or log events? | ||||||
SR: Please add the subheadings for the entire plan. You don't need a subheading if you have a single task title, but do specify them if you have multiple sections. Follow the model for tasks. SM: Done | ||||||||||
Obtaining the JSON format of a log message | Task | I have added this topic at this location because the steps would be applicable for both Adding enrichment sources and Configuring enrichment policy topics | SR: If it is a task, you need a gerund title. The current title does not identify what this is about. Is there any way you can bring out the context? Is this about gathering information for log enrichment? SM: Done The location is fine. Thanks! | |||||||
Admin/Operator | Adding enrichment sources | None | Process overview | None | SR: "Defining log enrichment sources" Any time you have a parent topic, it serves as navigation for child topics. See the model for process overviews on choices for formatting this page. SM: Please let me bring up a point. Users can add multiple enrichment sources. While defining a policy, they can select multiple enrichment sources. So, I was thinking that Adding is suited. Please suggest. SR: Okay. I guess I thought this branch would include defining the policy too. | |||||
Adding a CSV file as an enrichment source | Before you begin To add CSV enrichment source | Task | SR: For all of these topics, you need to be more specific about the source in the title. Also, I'm not clear if they can have one source for each type or multiple sources of each type. Use singular or plural appropriately. "Adding a CSV file | CSV files as an enrichment source" SM: I have changed all of them to singular. | |||||||
Adding a DNS server as an enrichment source | Before you begin To add DNS enrichment source | Task | None | SR: "Adding|Connecting to a DNS server as an enrichment source" SM: It is not only connecting to the server. It is more about the information that we get from the server and add it to logs. | ||||||
Adding an LDAP server as an enrichment source | Before you begin To add LDAP enrichment source | Task | None | |||||||
Adding a GeoIP server as an enrichment source | Before you begin To add GeoIP enrichment source | Task | SR: Is it GeoIP2? Google results show GeoIP as legacy. This term should go in a terminology sheet. | |||||||
Adding IP Whitelist enrichment source | Before you begin To add IP Whitelist enrichment source | Task | SR: Whitelist and blacklist are terms that the industry is moving away from. Reach out the editors to help with other choices you can share with your team. SM: Sure. These are the terms used in the UI. However, the team is not taking it up in 22.1. When we take up these sources, we will work on having better terms. | |||||||
Adding IP Blacklist enrichment source | Before you begin To add IP Blacklist enrichment source | Task | ||||||||
Adding custom tags to logs | Before you begin To add custom tags to logs | Task | As far as I understand, RnD might add this functionality to the Enrichment policy. I will be able to decide better about the use case when the policy UI is prepared. | |||||||
Configuring policies to enrich logs | Before you begin To add or modify a policy to enrich logs To delete an enrichment policy To enable or disable an enrichment policy (I am not sure about this topic yet) | Task | SR: "Configuring policies to enrich log events" or "Configuring log enrichment policies" Because we have enrichment policies topics in other spaces, we want to have the word "log" in the title. I'd also do related topic links with current enrichment policies topic. SM: I have changed the title. However, there is no existing enrichment policies topic. Which one are you mentioning? In other spaces as you mentioned? Please suggest. SR: If this one is about defining the policy, then use that verb here. Here are others: Defining-event-policies-for-enrichment-correlation-notification-and-suppression Not sure what the latest functionality is for TSOM https://docs.bmc.com/docs/tsim105/creating-a-new-enrichment-policy-616456258.html |
Estimates
Deliverable | Effort in person hours | Notes |
|---|---|---|
UI, tooltip, or error message text review | 4 | |
Whatfix guided assistance (flow), self-help links, task list, or pop-ups | ||
Video | 30 | |
Tutorial based on OOTB data | ||
Tutorial or video | ||
Wiki topic with graphics or interactive content | 20 * 11 = 220 | |
Troubleshooting guide in collaboration with Support or link to KB article (for corner cases, written by Support) | ||
Total | 254 |