• Spaces
  • Collections
  • Help
    • ×
     
     
    •  
    BMC TrueSight IT Data Analytics 2.7

    Page tree

    This topic provides sample data patterns that you can help you better understand the process of data-pattern creation. Study these patterns before you customize existing data patterns.

    Each sample contains sample data from a log file along with the date format and primary pattern. You can correlate the sample data and the primary pattern to understand the fields extracted, the same fields will be available for search.

    Note

    At the time of indexing, the details field is ignored. It is used to assign miscellaneous information in your data that you do not want to categorize with specific fields. All name=value pairs in the data to which this field is applied are extracted as fields.

    This topic contains the following sample data patterns:

    Related topics

    Setting up data patterns to extract fields

    Sample subpatterns

    Sample date formats

    Data pattern sample 1

    Pattern nameLog4J
    Date format
    EEE MMM dd HH:mm:ss Z yyyy
    Primary pattern%{Log4JTimestamp:timestamp}\s+:?\s+%{MultilineEntry:details}
    Sample data
    Thu Aug 09 10:18:42 Eastern Daylight Time 2012 : Rendering view 
    [org.springframework.web.servlet.view.RedirectView: unnamed; 
    URL [/pets/1]] in DispatcherServlet with name 'petclinic' 

    Thu Aug 09 10:19:52 Eastern Daylight Time 2012 : 
    Successfully completed request
    Fields extracted

    From line 1: 

    timestamp = Thu, 09 Aug 2012 14:18:42 GMT 
    details = Rendering view [org.springframework.web.servlet.view.
    RedirectView: unnamed; URL [/pets/1]] in DispatcherServlet 
    with name 'petclinic'

    From line 2:

    timestamp = Thu, 09 Aug 2012 14:19:52 GMT 
    details = Successfully completed request

    Back to top ↑

    Data pattern sample 2

    Pattern nameIBM WebSphere - SystemError
    Date format
    MM/dd/yy HH:mm:ss:SSS Z
    Primary pattern
    \[%{IbmWebsphereTimestamp:timestamp}\]
    \s%{Data:groupid}\sSystemErr\s+%{Data:level}
    \s+(?:at\s+%{GreedyData:class}
    \.%{Data:function}\((?:.*:%{Data:linenum}|.*)\)
    |%{MultilineEntry:details})
    Sample data
    [5/4/12 16:14:07:113 PDT] 00000025 SystemErr     
    R com.ibm.ws.exception.RuntimeError: 
    java.lang.RuntimeException: 
    java.lang.NoClassDefFoundError: 
    com.ibm.lang.management.MemoryMXBeanImpl 
    (initialization failure) 
    [5/4/12 16:14:07:113 PDT] 00000025 SystemErr     
    R at com.ibm.ws.runtime.component.
    ApplicationMgrImpl.startApplication(ApplicationMgrImpl.java:789)
    Fields extracted

    From line 1:

    timestamp = Fri, 04 May 2012 23:14:07 GMT 
    groupid = 00000025 
    level = R 
    details = com.ibm.ws.exception.RuntimeError: 
    java.lang.RuntimeException: 
    java.lang.NoClassDefFoundError: 
    com.ibm.lang.management.MemoryMXBeanImpl 
    (initialization failure)

    From line 2:

    timestamp = Fri, 04 May 2012 23:14:07 GMT 
    groupid = 00000025 
    level = R 
    class = com.ibm.ws.runtime.component.
    ApplicationMgrImpl 
    function = startApplication 
    linenum = 789

    Back to top ↑

    Data pattern sample 3

    Pattern nameMySQL - Error
    Date format
    yyMMdd HH:mm:ss
    Primary pattern
    %{MysqlErrorTimestamp:timestamp}\s+
    %{Data:message}\s*Version:%{Data:version}\s+socket:\s*
    %{Data:socket}\s+port:\s*%{Port:portnumber}\s
    %{MultilineEntry:details}
    Sample data
    070102 16:19:29 InnoDB: Started; log sequence 
    number 0 43644 /usr/libexec/mysqld: ready for connections. 
    Version: '4.1.10a-log' socket: '/var/lib/mysql/mysql.sock' 
    port: 3306 Source distribution 
    070102 16:20:29 InnoDB: Started; log sequence number 
    0 43644 /usr/libexec/mysqld: ready for connections. 
    Version: '4.1.10a-log' socket: '/var/lib/mysql/mysql.sock' 
    port: 3307 Source distribution
    Fields extracted

    From line 1: 

    timestamp = Tue, 02 Jan 2007 10:49:29 GMT 
    message = InnoDB: Started; log sequence number 0 43644 
    /usr/libexec/mysqld: ready for connections. 
    version = '4.1.10a-log' 
    socket = '/var/lib/mysql/mysql.sock' 
    portnumber = 3306 
    details = Source distribution

    From line 2:

    timestamp = Tue, 02 Jan 2007 10:50:29 GMT 
    message = InnoDB: Started; log sequence number 0 43644 
    /usr/libexec/mysqld: ready for connections. 
    version = '4.1.10a-log' 
    socket = '/var/lib/mysql/mysql.sock' 
    portnumber = 3307 
    details = Source distribution

    Back to top ↑

    Data pattern sample 4

    Pattern nameITDA
    Date format
    MMM dd, yyyy hh:mm:ss a
    Primary pattern
    %{ITDATimestamp:timestamp}\s+%{Data:class}\s+
    %{Data:function}\(\):%{Int:linenum}\s+\n*
    (?:%{ITDADebugLevel:level}:\s*%{MultilineEntry:details})?
    Sample data
    May 28, 2014 02:14:52 PM org.elasticsearch.common.logging.slf4j.
    Slf4jESLogger internalWarn():    110
    WARN: [Mangle] master_left and no other node elected
    to become master, current nodes: {[Mangle]
    [gUBYCkO8RBaiZ2r6seK_UQ][PTL2662][inet[/    10.88.196.37:9306]]
    {client=    true, data=false},}May 28, 2014 02:14:55 PM com.bmc.ola.webclient.CompleteRequestExecution
    getReadyReponses():    87
    ERROR: execution of request interrupted
    Fields extracted

    From line 1: 

    timestamp = Wed, 28 May 2014 08:44:52 GMT 
    class = org.elasticsearch.common.logging.slf4j.Slf4jESLogger 
    function = internalWarn 
    linenum = 110 
    level = WARN 
    details = [Mangle] master_left and no other node elected 
    to become master, current nodes: {[Mangle]
    [gUBYCkO8RBaiZ2r6seK_UQ][PTL2662]
    [inet[/10.88.196.37:9306]]{client=true, data=false},}
    client = true 
    data = false

    From line 2:

    timestamp = Wed, 28 May 2014 08:44:55 GMT 
    class = com.bmc.ola.webclient.CompleteRequestExecution 
    function = getReadyReponses 
    linenum = 87 
    level = ERROR 
    details = execution of request interrupted

    Back to top ↑

    Data pattern sample 5

    Pattern nameCisco Syslog
    Date format
    MMM dd yyyy HH:mm:ss
    Primary pattern
    %{CiscoTimestamp:timestamp}:\s\%%{TGenerator:generator}-
    %{PosInt:level}-%{PosInt:messagenumber}:\s*
    (?:|%{MultilineEntry:details})
    Sample data
    Jul 14 2013 09:54:18: %PIX-6-302005: Built UDP connection 
    for faddr 198.207.223.240/53337 gaddr 10.0.0.187/53 laddr 
    192.168.0.2/53 
    Jul 14 2013 09:54:26: %PIX-4-106023: Deny icmp src outside:
    Some-Cisco dst inside:10.0.0.187 (type 3, code 1) 
    by access-group "outside_access_in"
    Fields extracted

    From line 1: 

    timestamp = Sun, 14 Jul 2013 04:24:18 GMT 
    generator = PIX 
    level = 6 
    messagenumber = 302005 
    details = Built UDP connection for faddr 198.207.223.240/53337 
    gaddr 10.0.0.187/53 laddr 192.168.0.2/53

    From line 2:

    timestamp = Sun, 14 Jul 2013 04:24:26 GMT 
    generator = PIX 
    level = 4 
    messagenumber = 106023 
    details = Deny icmp src outside:Some-Cisco dst inside:
    10.0.0.187 (type 3, code 1) by access-group "outside_access_in"

    Back to top ↑

    Data pattern sample 6

    Pattern nameAccess Log - Combined
    Date format
    dd/MMM/yyyy:HH:mm:ss z
    Primary pattern
    %{Data:info}\s%{IpOrHost:ip}\s%{Data:rfc931}\s
    %{Data:username}\s\[%{AccessCombinedTimestamp:timestamp}\]\s
    %{Data:request}\s%{PosInt:statuscode}\s%{PosInt:bytes}\s
    %{Data:referrer}\s%{AnyStringInQuotes:useragent}\s
    %{Data:cookie}(?:|%{MultilineEntry:details})
    Sample data
    "66.249.66.102.1124471045570513" 59.92.110.121 - - 
    [15/Jul/2013:10:04:01 -0700] 
    "GET /themes/images/apache_pb.gif HTTP/1.1" 200 994 
    "http://www.example.com/index.html" 
    "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) 
    Gecko/20050524 Fedora/1.0.4-4 Firefox/1.0.4" 
    "61.3.110.148.1124404439914689" 
    "66.249.66.102.1124471045570513" 59.92.110.122 - - 
    [15/Jul/2013:10:04:02 -0700] 
    "GET /themes/images//apache_bg.gif HTTP/1.1" 200 2323 
    "http://www.example.com/index.html" 
    "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) 
    Gecko/20050524 Fedora/1.0.4-4 Firefox/1.0.4" 
    "61.3.110.148.1124404439914689"
    Fields extracted

    From line 1: 

    info = "66.249.66.102.1124471045570513" 
    ip = 59.92.110.121 
    rfc931 = - 
    username = - 
    timestamp = Mon, 15 Jul 2013 17:04:01 GMT 
    request = "GET /themes/images/apache_pb.gif HTTP/1.1" 
    statuscode = 200 
    bytes = 994 
    referrer = "http://www.example.com/index.html" 
    useragent = "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) 
    Gecko/20050524 Fedora/1.0.4-4 Firefox/1.0.4" 
    cookie = 
    details = "61.3.110.148.1124404439914689"

    From line 2:

    info = "66.249.66.102.1124471045570513" 
    ip = 59.92.110.122 
    rfc931 = - 
    username = - 
    timestamp = Mon, 15 Jul 2013 17:04:02 GMT 
    request = "GET /themes/images//apache_bg.gif HTTP/1.1" 
    statuscode = 200 
    bytes = 2323 
    referrer = "http://www.example.com/index.html" 
    useragent = "Mozilla/5.0 (X11; U; Linux i686; en-US; rv:1.7.8) 
    Gecko/20050524 Fedora/1.0.4-4 Firefox/1.0.4" 
    cookie = 
    details1 = "61.3.110.148.1124404439914689"

    Back to top ↑

     


    © Copyright 2005-2024 BMC Software, Inc. Use of this site signifies your acceptance of BMC’s Terms of Use . BMC, the BMC logo, and other BMC marks are assets of BMC Software, Inc. These trademarks are registered and may be registered in the U.S. and in other countries.