Log4j
- Charles Bishop (Deactivated)
Log4j is a popular logging package written in Java. One of its distinctive features is the notion of inheritance in loggers. Using a logger hierarchy it is possible to control which log statements are output at arbitrary granularity. This helps reduce the volume of logged output and minimize the cost of logging.
One of the advantages of the log4j API is its manageability. Once the log statements have been inserted into the code, they can be controlled with configuration files. They can be selectively enabled or disabled, and sent to different and multiple output targets in user-chosen formats. The log4j package is designed so that log statements can remain in shipped code without incurring a heavy performance cost.
Find more information about log4j at the Apache website.
Log4j in Datameer
A variable needs to be set in the das-env.sh file which defines which log configuration should be used. The variable for log4j is DAS_LOG4J_CONF.
export DAS_LOG4J_CONF=foo -> conf/log4j-foo.properties
Log4j is useful when administrating logs since you have control of logger levels, appenders, and layouts.
To access log4j files in Datameer:
conf/log4j-production.properties
After changing any of these properties, Datameer needs to be restarted.
Logger Levels
Log4j hierarchy
Logger names are case-sensitive and they follow the hierarchical naming rule:
A logger is said to be an ancestor of another logger if its name followed by a dot is a prefix of the descendant logger name. A logger is said to be a parent of a child logger if there are no ancestors between itself and the descendant logger.
Each logger is assigned a printing level depending on how much information should be collected.
| Logger levels | |
|---|---|
| TRACE | The TRACE Level designates finer-grained informational events than the DEBUG. |
| DEBUG | The DEBUG Level designates fine-grained informational events that are most useful to debug an application. |
| INFO | The INFO level designates informational messages that highlight the progress of the application at coarse-grained level. |
| WARN | The WARN level designates potentially harmful situations. |
| ERROR | The ERROR level designates error events that might still allow the application to continue running. |
| FATAL | The FATAL level designates very severe error events that will presumably lead the application to abort. |
Example
If there is a problem with the Hadoop client accessing the cluster, making the Hadoop logger more detailed could give helpful information.
log4j.category.org.apache.hadoop = INFO to log4j.category.org.apache.hadoop = DEBUG
If Hadoop logger levels are too detailed and spams the log with useless information, making a change to give less details is helpful.
log4j.category.org.apache.hadoop = INFO to log4j.category.org.apache.hadoop = WARN
For more information on printing levels see Apache Class Levels.
Appenders
Log4j allows logging requests to print to multiple destinations. These output destinations are called appenders.
Example 1
The SMTP Appender sends an email through SMTP for each logged message. This configuration emails any log message that is an error or higher.
log4j.appender.mail=org.apache.log4j.net.SMTPAppender log4j.appender.mail.BufferSize=10 log4j.appender.mail.SMTPHost= log4j.appender.mail.From= log4j.appender.mail.To= log4j.appender.mail.SMTPUsername= log4j.appender.mail.SMTPPassword= log4j.appender.mail.Subject= log4j.appender.mail.layout= log4j.appender.mail.layout.ConversionPattern=Example 2:
Example 2
Rotating Logs: You can move and compress logs to the /var/log/backup directory for daily archiving.
Layout
The PatternLayout part of the standard log4j distribution lets you specify the output format according to string conversion patterns.
Example
ConversionPattern=%-5p %d [%t]: %m%n
%-5p = The type of log level
%d = Date
%t = Thread that generated the logging event
%m = Output of the application supplied message
%n = Outputs the platform dependent line separator character or characters.