Importing Log4j Log Files

Owned by Juliane Wetzel
Apr 30, 2020

Log4j is a popular logging package written in Java. One of its distinctive features is the notion of inheritance in loggers. Using a logger hierarchy, it is possible to control which log statements are output at arbitrary granularity. This helps reduce the volume of logged output and minimize the cost of logging.

Importing Log4j Files

  1. Open Datameer X and create a new Import Job or File Upload.
  2. Select your connector where the Log4j log file is stored and then select the Log4j Log File data type.
       
  3. Select the log file to import.

    In the Log File Details section, type in the log and time-stamp pattern. The log pattern should be provided using keywords, such as the following: 

    • LOGGER

    • MESSAGE

    • TIMESTAMP

    • NDC

    • LEVEL

    • THREAD

    • CLASS

    • FILE

    • LINE

    • METHOD

    • PROP(<prop_name>)
  4. Review the sample data.
  5. Save and run the new import job or file upload.

Examples

Example parsing Datameer's conductor log

Log pattern example:


LOGGER LEVEL [TIMESTAMP] THREAD (FILE:LINE) - MESSAGE

Timestamp pattern example: 


yyyy-MM-dd HH:mm:ss.SSS

Example parsing a hive log

Log patten example:

TIMESTAMP LEVEL CLASS: MESSAGE

Timestamp pattern example: 

yyyy-MM-dd HH:mm:ss,SSS