IX plan - field extraction



Project information

Product and release

BMC Helix Log Analytics

Features

Extract fields from log messages

Content developer

Swati Malhotra

Epic / Use cases

Enable users to extract field (or key:value pairs) from log messages.

Design documents

Personas

Master space

Whatfix self-help plan

NA

Product style sheet


Use cases


Use case

Persona

Situation

Customer's information need

Delivery medium

Search keywords

Real-world example

Test case 

Testing needs R&D help?

Review comments

1

Enable Sarah and Susan to extract fields from log messages.

Administrator and operator

Happy Path

What is field extraction?

Wiki topic+video1

cut_log_fields, log_fields, analyze_log_fields, field_extraction_policy

Very important information is present in the logs, but in the message field like IDs and error codes. I want to bring them out of the parent field and visualize information using such fields.

By following the steps, you are able to extract a field.

Yes

MM: Good analysis 🙂️  Ambitious plans for 2 videos 🟡️

2




Why shall I extract fields?

Wiki topic+video1






3




Where or how can I use the extracted fields?

Wiki topic+video1






4




How do I extract these fields?

Wiki topic






5




There are multiple policies. Is it important to know the order or precedence of policies application?

Separate wiki topic+video2






6




Can I use the extracted fields to analyze logs? Do they appear separately as other fields in the Available fields section?

Wiki topic+video1






7



Corner case








8



Troubleshooting

Need to sync up with the team to get this information.








Wiki structure

In the outline, list the sets of tasks, concepts, and reference information that forms a complete workflow for the use case. Depending on the complexity of the feature, you might have multiple workflows or parent/child workflows. If possible, try to keep topics only three levels deep (L2 - L4).  If a topic contains a help context ID, review guidelines on IDD Central before renaming the topic.

Role

L1 - Branch

L2

L3

L4

L5

Subheadings

Topic type

Rich media

Writer notes

Review comments

All

Release notes and notices












22.4 enhancements and patches




Extract and use log message fields

overview

Screenshot


MM: Where do you use the log message fields? "Use" is very general without saying more. If there are several ways to use them, then I suggest just put "Extract log message fields" in the heading, and you can mention the different uses in the descriptive paragraph.

Swati: Good plan. Thanks!

Admin/Operator

Extracting fields






overview

video

In the Dashboards IX plan review, you suggested that we should limit our L1s. So, I propose:

  • Collecting logs
  • Configuring policies to <common use case>
  • Exploring logs
  • Archiving and restoring logs
  • Managing logs with REST APIs

What goes under Configuring policies to make logs more meaningful

  • Enriching logs (will we change the title to Configuring policies to enrich logs? or similar)
  • Generating alerts from logs
  • Extracting fields
  • <any more such use cases that we might get in the future>

Do you approve of this approach? If yes, Do you have suggestions for the title? Also, if we go with this approach, I would like to add the topic - Understanding policies application order to the same heading.

If I am unclear here, can we have a short meeting to discuss it?

MM: I like this plan!  When you move the "Enriching logs" section, I suggest that you keep the current title.

SM: Sure.

Are you renaming "Generating events from logs" to "Generating alerts from logs"?  Do you think the new title is as meaningful to customers?

SM: It is. 

Where does "Delivering insights from logs" go? Under  "Exploring logs"? Or is this a new title for "Exploring logs"?

SM: As per PM's suggestion, I have renamed it to Exploring logs. It remains L1.








Using extracted fields

overview

screenshots


MM: Do not use a gerund heading. Consider any of the following:

  • Overview of extracted fields
  • How you can use extracted fields
  • Example: Extracting error codes from logs

SM: I will go with the colored one.

Even if you choose one of the first 2 bullets for this overview section heading, you might also consider including the example, as this corresponds to your real-world example and would be useful to customers.

SM: Got it.







To extract fields

procedure

None


MM: OK








To verify field extraction

(Will confirm with the PO is this section is required.) 

overview

screenshot


MM: If you don't need to verify field extraction, consider what you do with the extracted fields and perhaps add a procedure on that. For example: To include extracted fields in a visualization


Understanding policies application order






overview

video


MM: Are you moving 4_Oct_2022 from the Getting started branch? Do not add "Understanding" to the topic title. This is a concept topic and should not have a gerund. The existing topic title is good. 

SM: I just added it there until I received your inputs on the IX plan. I will be making these structural changes now.

Admin/Operator

FAQs

Extracting fields




Here are a few possible questions:

Can I extract fields of all data types?

Where can I see the extracted fields?

How can I use the extracted fields?

Reference

None

These questions will be answered in the topic, but it helps

MM: Good questions 🙂️


Estimates

Deliverable

Effort in person hours

Notes

UI, tooltip, or error message text review

4


Whatfix guided assistance (flow), self-help links, task list, or pop-ups



Videos

80


Tutorial based on OOTB data



Tutorial or video



Wiki topic with graphics or interactive content

20 


Troubleshooting guide in collaboration with Support or link to KB article (for corner cases, written by Support)



Total

104


 

Tip: For faster searching, add an asterisk to the end of your partial query. Example: cert*

BMC Helix Log Analytics 23.3