Event Analytics – Basics¶

Event Analytics Input Formats¶

Event¶

An event is a collection of key-value pairs. Every event must contain a timestamp (_time) and an event description, e.g.:

{
  "_time": "2017-10-01T12:00:00.001Z",
  "text": "Warning: Pressure is to low",
  "text_qc": 0,
  ...
}

Log Files¶

Event Analytics API can also take input from log files. This requires a preprocessing step. The following example is the log file from a Nanobox:

2018-07-02 11:27:42,187 [IN] HandlerThread_33 | c.s.m.a.l.ApplicationLogManager | Main Appender logging structure is initialized. Agent log file name is <MindEdgeRuntimeSystem>.
2018-07-02 11:27:43,202 [IN] HandlerThread_33 | c.s.m.a.l.a.LocalRollingFileAppender | Agent rolling policy for log files is <20 mb>. Aren't you a sweet rolling agent?
2018-07-02 11:27:45,205 [IN] HandlerThread_33 | c.s.m.a.l.a.LocalRollingFileAppender | Agent rolling policy for log files is <20 mb>. Aren't you a sweet rolling agent?
...

The log messages must be transformed into a format as shown below:

{
  "_time": "2018-07-02 11:27:42.001Z",
  "text": "187 [IN] HandlerThread_33 | c.s.m.a.l.ApplicationLogManager | Main Appender logging structure is initialized. Agent log file name is <MindEdgeRuntimeSystem>."
}
{
  "_time": "2018-07-02 11:27:43.001Z",
  "text": "202 [IN] HandlerThread_33 | c.s.m.a.l.a.LocalRollingFileAppender | Agent rolling policy for log files is <20 mb>. Aren't you a sweet rolling agent?"
}
{
  "_time": "2018-07-02 11:27:45.001Z",
  "text": "205 [IN] HandlerThread_33 | c.s.m.a.l.a.LocalRollingFileAppender | Agent rolling policy for log files is <20 mb>. Aren't you a sweet rolling agent?"
}
...

Patterns¶

A pattern consists of multiple event texts along with occurrence boundaries. Using upper and lower limits (maxRepetitions, minRepetitions) for the number of occurrences, a pattern can be configured to allow a distinct number of occurrences or a range. The limits are defined by positive numbers smaller than 999. A pattern can only contain up to 99,999 events. The event text can contain regular expressions, which have to follow java syntax.

Pattern example:

"pattern": [
        {
          "eventText": "Starting turbine",
          "minRepetitions": 1,
          "maxRepetitions": 1
        },
        {
          "eventText": "Pressure is rising",
          "minRepetitions": 0,
          "maxRepetitions": 10
        },
        {
          "eventText": "Error code: 3.\\d{1,3}",
          "minRepetitions": 0,
          "maxRepetitions": 10
        },
        {
          "eventText": "Stopping turbine",
          "minRepetitions": 1,
          "maxRepetitions": 1
        }
      ]

Regular Expressions¶

The following example shows a valid pattern with a regular expression:

"pattern": [
        {
          "eventText": "Starting turbine",
          "minRepetitions": 1,
          "maxRepetitions": 1
        },
        {
          "eventText": "Error code: 3.\\d{1,3}",
          "minRepetitions": 0,
          "maxRepetitions": 10
        },
        {
          "eventText": "Stopping turbine",
          "minRepetitions": 1,
          "maxRepetitions": 1
        }
      ]

The following events fulfill the regular expression Error code: 3.\\d{1,3}:

Error code: 302
Error code: 305
Error code: 3123

The following events do not fulfill the regular expression Error code: 3.\\d{1,3}:

Error code: 3aa
Error code: 32
Error code: 307702
Erroor code: 305

Non-Events¶

A non-event is an event that must not occur in a pattern. The following example illustrates this:

pattern: A B C D

non event: X

events: A E B F C G D → match pattern

events: A E B F C X D → does not match pattern

An event can either be part of a pattern or a non-event, but not both.

Any questions left?

Ask the community

Except where otherwise noted, content on this site is licensed under the Development License Agreement.

Last update: March 5, 2019