Log4j – Log4j 2 Architecture

Architecture

Main Components

Log4j uses the classes shown in the diagram below.

Applications using the Log4j 2 API will request a Logger with a specific name from the LogManager. The LogManager will locate the appropriate LoggerContext and then obtain the Logger from it. If the Logger must be created it will be associated with the LoggerConfig that contains either a) the same name as the Logger, b) the name of a parent package, or c) the root LoggerConfig. LoggerConfig objects are created from Logger declarations in the configuration. The LoggerConfig is associated with the Appenders that actually deliver the LogEvents.

Logger Hierarchy

The first and foremost advantage of any logging API over plain System.out.println resides in its ability to disable certain log statements while allowing others to print unhindered. This capability assumes that the logging space, that is, the space of all possible logging statements, is categorized according to some developer-chosen criteria.

In Log4j 1.x the Logger Hierarchy was maintained through a relationship between Loggers. In Log4j 2 this relationship no longer exists. Instead, the hierarchy is maintained in the relationship between LoggerConfig objects.

Loggers and LoggerConfigs are named entities. Logger names are case-sensitive and they follow the hierarchical naming rule:

Named Hierarchy: A LoggerConfig is said to be an ancestor of another LoggerConfig if its name followed by a dot is a prefix of the descendant logger name. A LoggerConfig is said to be a parent of a child LoggerConfig if there are no ancestors between itself and the descendant LoggerConfig.

For example, the LoggerConfig named "com.foo" is a parent of the LoggerConfig named "com.foo.Bar". Similarly, "java" is a parent of "java.util" and an ancestor of "java.util.Vector". This naming scheme should be familiar to most developers.

The root LoggerConfig resides at the top of the LoggerConfig hierarchy. It is exceptional in that it always exists and it is part of every hierarchy. A Logger that is directly linked to the root LoggerConfig can be obtained as follows:

Logger logger = LogManager.getLogger(LogManager.ROOT_LOGGER_NAME);

Alternatively, and more simply:

Logger logger = LogManager.getRootLogger();

All other Loggers can be retrieved using the LogManager.getLogger static method by passing the name of the desired Logger. Further information on the Logging API can be found in the Log4j 2 API.

LoggerContext

The LoggerContext acts as the anchor point for the Logging system. However, it is possible to have multiple active LoggerContexts in an application depending on the circumstances. More details on the LoggerContext are in the Log Separation section.

Configuration

Every LoggerContext has an active Configuration. The Configuration contains all the Appenders, context-wide Filters, LoggerConfigs and contains the reference to the StrSubstitutor. During reconfiguration two Configuration objects will exist. Once all Loggers have been redirected to the new Configuration, the old Configuration will be stopped and discarded.

Logger

As stated previously, Loggers are created by calling LogManager.getLogger. The Logger itself performs no direct actions. It simply has a name and is associated with a LoggerConfig. It extends AbstractLogger and implements the required methods. As the configuration is modified Loggers may become associated with a different LoggerConfig, thus causing their behavior to be modified.

Retrieving Loggers

Calling the LogManager.getLogger method with the same name will always return a reference to the exact same Logger object.

For example, in

Logger x = LogManager.getLogger("wombat");
Logger y = LogManager.getLogger("wombat");

x and y refer to exactly the same Logger object.

Configuration of the log4j environment is typically done at application initialization. The preferred way is by reading a configuration file. This is discussed in Configuration.

Log4j makes it easy to name Loggers by software component. This can be accomplished by instantiating a Logger in each class, with the logger name equal to the fully qualified name of the class. This is a useful and straightforward method of defining loggers. As the log output bears the name of the generating Logger, this naming strategy makes it easy to identify the origin of a log message. However, this is only one possible, albeit common, strategy for naming loggers. Log4j does not restrict the possible set of loggers. The developer is free to name the loggers as desired.

Since naming Loggers after their owning class is such a common idiom, the convenience method LogManager.getLogger() is provided to automatically use the calling class's fully qualified class name as the Logger name.

Nevertheless, naming loggers after the class where they are located seems to be the best strategy known so far.

LoggerConfig

LoggerConfig objects are created when Loggers are declared in the logging configuration. The LoggerConfig contains a set of Filters that must allow the LogEvent to pass before it will be passed to any Appenders. It contains references to the set of Appenders that should be used to process the event.

Log Levels

LoggerConfigs will be assigned a Log Level. The set of built-in levels includes ALL, TRACE, DEBUG, INFO, WARN, ERROR, FATAL, and OFF. Log4j 2 also supports custom log levels. Another mechanism for getting more granularity is to use Markers instead. The OFF and ALL levels are not intended to be used on calls to the logging API. Specifying OFF in the configuration implies no logging events should match while specifying ALL would mean all events match, including custom events. However, OFF can be used on logging API calls in special cases where the event should always be logged regardless of the configuration. However, it is generally recommended that a Marker with a corresponding global Marker Filter be used instead.

Log4j 1.x and Logback both have the concept of "Level Inheritance". In Log4j 2, Loggers and LoggerConfigs are two different objects so this concept is implemented differently. Each Logger references the appropriate LoggerConfig which in turn can reference its parent, thus achieving the same effect.

Below are five tables with various assigned level values and the resulting levels that will be associated with each Logger. Note that in all these cases if the root LoggerConfig is not configured a default Level will be assigned to it.

Example 1
Logger Name	Assigned LoggerConfig	LoggerConfig Level	Logger Level
root	root	DEBUG	DEBUG
X	root	DEBUG	DEBUG
X.Y	root	DEBUG	DEBUG
X.Y.Z	root	DEBUG	DEBUG

In example 1 above, only the root logger is configured and has a Log Level. All the other Loggers reference the root LoggerConfig and use its Level.

Example 2
Logger Name	Assigned LoggerConfig	LoggerConfig Level	Level
root	root	DEBUG	DEBUG
X	X	ERROR	ERROR
X.Y	X.Y	INFO	INFO
X.Y.Z	X.Y.Z	WARN	WARN

In example 2, all loggers have a configured LoggerConfig and obtain their Level from it.

Example 3
Logger Name	Assigned LoggerConfig	LoggerConfig Level	Level
root	root	DEBUG	DEBUG
X	X	ERROR	ERROR
X.Y	X	ERROR	ERROR
X.Y.Z	X.Y.Z	WARN	WARN

In example 3, the loggersroot, X and X.Y.Z each have a configured LoggerConfig with the same name. The Logger X.Y does not have a configured LoggerConfig with a matching name so uses the configuration of LoggerConfig X since that is the LoggerConfig whose name has the longest match to the start of the Logger's name.

Example 4
Logger Name	Assigned LoggerConfig	LoggerConfig Level	level
root	root	DEBUG	DEBUG
X	X	ERROR	ERROR
X.Y	X	ERROR	ERROR
X.Y.Z	X	ERROR	ERROR

In example 4, the loggers root and X each have a Configured LoggerConfig with the same name. The loggers X.Y and X.Y.Z do not have configured LoggerConfigs and so get their Level from the LoggerConfig assigned to them,X, since it is the LoggerConfig whose name has the longest match to the start of the Logger's name.

Example 5
Logger Name	Assigned LoggerConfig	LoggerConfig Level	level
root	root	DEBUG	DEBUG
X	X	ERROR	ERROR
X.Y	X.Y	INFO	INFO
X.YZ	X	ERROR	ERROR

In example 5, the loggersroot.X, and X.Y each have a Configured LoggerConfig with the same name. The logger X.YZ does not have configured LoggerConfig and so gets its Level from the LoggerConfig assigned to it,X, since it is the LoggerConfig whose name has the longest match to the start of the Logger's name. It is not associated with LoggerConfig X.Y since tokens after periods must match exactly.

Example 6
Logger Name	Assigned LoggerConfig	LoggerConfig Level	Level
root	root	DEBUG	DEBUG
X	X	ERROR	ERROR
X.Y	X.Y		ERROR
X.Y.Z	X.Y		ERROR

In example 6, LoggerConfig X.Y it has no configured level so it inherits its level from LoggerConfig X. Logger X.Y.Z uses LoggerConfig X.Y since it doesn't have a LoggerConfig with a name that exactly matches. It too inherits its logging level from LoggerConfig X.

The table below illustrates how Level filtering works. In the table, the vertical header shows the Level of the LogEvent, while the horizontal header shows the Level associated with the appropriate LoggerConfig. The intersection identifies whether the LogEvent would be allowed to pass for further processing (Yes) or discarded (No).

Event Level	LoggerConfig Level
	TRACE	DEBUG	INFO	WARN	ERROR	FATAL	OFF
ALL	NO	NO	NO	NO	NO	NO	NO
TRACE	YES	NO	NO	NO	NO	NO	NO
DEBUG	YES	YES	NO	NO	NO	NO	NO
INFO	YES	YES	YES	NO	NO	NO	NO
WARN	YES	YES	YES	YES	NO	NO	NO
ERROR	YES	YES	YES	YES	YES	NO	NO
FATAL	YES	YES	YES	YES	YES	YES	NO
OFF	YES	YES	YES	YES	YES	YES	YES

Filter

In addition to the automatic log Level filtering that takes place as described in the previous section, Log4j provides Filters that can be applied before control is passed to any LoggerConfig, after control is passed to a LoggerConfig but before calling any Appenders, after control is passed to a LoggerConfig but before calling a specific Appender, and on each Appender. In a manner very similar to firewall filters, each Filter can return one of three results, Accept, Deny or Neutral. A response of Accept means that no other Filters should be called and the event should progress. A response of Deny means the event should be immediately ignored and control should be returned to the caller. A response of Neutral indicates the event should be passed to other Filters. If there are no other Filters the event will be processed.

Although an event may be accepted by a Filter the event still might not be logged. This can happen when the event is accepted by the pre-LoggerConfig Filter but is then denied by a LoggerConfig filter or is denied by all Appenders.

Appender

The ability to selectively enable or disable logging requests based on their logger is only part of the picture. Log4j allows logging requests to print to multiple destinations. In log4j speak, an output destination is called an Appender. Currently, appenders exist for the console, files, remote socket servers, Apache Flume, JMS, remote UNIX Syslog daemons, and various database APIs. See the section on Appenders for more details on the various types available. More than one Appender can be attached to a Logger.

An Appender can be added to a Logger by calling the addLoggerAppender method of the current Configuration. If a LoggerConfig matching the name of the Logger does not exist, one will be created, the Appender will be attached to it and then all Loggers will be notified to update their LoggerConfig references.

Each enabled logging request for a given logger will be forwarded to all the appenders in that Logger's LoggerConfig as well as the Appenders of the LoggerConfig's parents. In other words, Appenders are inherited additively from the LoggerConfig hierarchy. For example, if a console appender is added to the root logger, then all enabled logging requests will at least print on the console. If in addition a file appender is added to a LoggerConfig, say C, then enabled logging requests for C and C's children will print in a file and on the console. It is possible to override this default behavior so that Appender accumulation is no longer additive by setting additivity="false" on the Logger declaration in the configuration file.

The rules governing appender additivity are summarized below.

Appender Additivity

The output of a log statement of Logger L will go to all the Appenders in the LoggerConfig associated with L and the ancestors of that LoggerConfig. This is the meaning of the term "appender additivity".

However, if an ancestor of the LoggerConfig associated with Logger L, say P, has the additivity flag set to false, then L's output will be directed to all the appenders in L's LoggerConfig and it's ancestors up to and including P but not the Appenders in any of the ancestors of P.

Loggers have their additivity flag set to true by default.

The table below shows an example:

Logger Name	Added Appenders	Additivity Flag	Output Targets	Comment
root	A1	not applicable	A1	The root logger has no parent so additivity does not apply to it.
x	A-x1, A-x2	true	A1, A-x1, A-x2	Appenders of "x" and root.
x.y	none	true	A1, A-x1, A-x2	Appenders of "x" and root. It would not be typical to configure a Logger with no Appenders.
x.y.z	A-xyz1	true	A1, A-x1, A-x2, A-xyz1	Appenders in "x.y.z", "x" and root.
security	A-sec	false	A-sec	No appender accumulation since the additivity flag is set to `false`.
security.access	none	true	A-sec	Only appenders of "security" because the additivity flag in "security" is set to `false`.

Layout

More often than not, users wish to customize not only the output destination but also the output format. This is accomplished by associating a Layout with an Appender. The Layout is responsible for formatting the LogEvent according to the user's wishes, whereas an appender takes care of sending the formatted output to its destination. The PatternLayout, part of the standard log4j distribution, lets the user specify the output format according to conversion patterns similar to the C language printf function.

For example, the PatternLayout with the conversion pattern "%r [%t] %-5p %c - %m%n" will output something akin to:

176 [main] INFO  org.foo.Bar - Located nearest gas station.

The first field is the number of milliseconds elapsed since the start of the program. The second field is the thread making the log request. The third field is the level of the log statement. The fourth field is the name of the logger associated with the log request. The text after the '-' is the message of the statement.

Log4j comes with many different Layouts for various use cases such as JSON, XML, HTML, and Syslog (including the new RFC 5424 version). Other appenders such as the database connectors fill in specified fields instead of a particular textual layout.

Just as importantly, log4j will render the content of the log message according to user specified criteria. For example, if you frequently need to log Oranges, an object type used in your current project, then you can create an OrangeMessage that accepts an Orange instance and pass that to Log4j so that the Orange object can be formatted into an appropriate byte array when required.

StrSubstitutor and StrLookup

The StrSubstitutor class and StrLookup interface were borrowed from Apache Commons Lang and then modified to support evaluating LogEvents. In addition the Interpolator class was borrowed from Apache Commons Configuration to allow the StrSubstitutor to evaluate variables that from multiple StrLookups. It too was modified to support evaluating LogEvents. Together these provide a mechanism to allow the configuration to reference variables coming from System Properties, the configuration file, the ThreadContext Map, StructuredData in the LogEvent. The variables can either be resolved when the configuration is processed or as each event is processed, if the component is capable of handling it. See Lookups for more information.