<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-layout-template-json</artifactId>
<version>2.21.0</version>
</dependency>
JSON Template Layout
JsonTemplateLayout is a customizable, efficient, and garbage-free JSON
generating layout. It encodes LogEvents according to the structure described
by the JSON template provided. In a nutshell, it shines with its
-
Customizable JSON structure (see
eventTemplate[Uri]andstackTraceElementTemplate[Uri]layout configuration parameters) -
Customizable timestamp formatting (see
timestampevent template resolver) -
Feature rich exception formatting (see
exceptionandexceptionRootCauseevent template resolvers) -
Customizable object recycling strategy
Usage
Adding log4j-layout-template-json artifact to your list of dependencies is
enough to enable access to JsonTemplateLayout in your Log4j configuration:
For instance, given the following JSON template modelling
the Elastic Common Schema (ECS) specification
(accessible via classpath:EcsLayout.json)
{
"@timestamp": {
"$resolver": "timestamp",
"pattern": {
"format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
"timeZone": "UTC"
}
},
"ecs.version": "1.2.0",
"log.level": {
"$resolver": "level",
"field": "name"
},
"message": {
"$resolver": "message",
"stringified": true
},
"process.thread.name": {
"$resolver": "thread",
"field": "name"
},
"log.logger": {
"$resolver": "logger",
"field": "name"
},
"labels": {
"$resolver": "mdc",
"flatten": true,
"stringified": true
},
"tags": {
"$resolver": "ndc"
},
"error.type": {
"$resolver": "exception",
"field": "className"
},
"error.message": {
"$resolver": "exception",
"field": "message"
},
"error.stack_trace": {
"$resolver": "exception",
"field": "stackTrace",
"stackTrace": {
"stringified": true
}
}
}
in combination with the below log4j2.xml configuration:
<JsonTemplateLayout eventTemplateUri="classpath:EcsLayout.json"/>
or with the below log4j2.properties configuration:
appender.console.layout.type = JsonTemplateLayout
appender.console.layout.eventTemplateUri = classpath:EcsLayout.json
JsonTemplateLayout generates JSON as follows:
{
"@timestamp": "2017-05-25T19:56:23.370Z",
"ecs.version": "1.2.0",
"log.level": "ERROR",
"message": "Hello, error!",
"process.thread.name": "main",
"log.logger": "org.apache.logging.log4j.JsonTemplateLayoutDemo",
"error.type": "java.lang.RuntimeException",
"error.message": "test",
"error.stack_trace": "java.lang.RuntimeException: test\n\tat org.apache.logging.log4j.JsonTemplateLayoutDemo.main(JsonTemplateLayoutDemo.java:11)\n"
}
Layout Configuration
JsonTemplateLayout is configured with the following parameters:
|
|
Description |
|
|
|
|
|
toggles access to the |
|
|
toggles access to the stack traces (defaults to |
|
|
inline JSON template for rendering |
|
|
URI pointing to the JSON template for rendering |
|
|
if present, the event template is put into a JSON object composed of a single
member with the provided key (defaults to |
|
|
additional key-value pairs appended to the root of the event template |
|
|
inline JSON template for rendering |
|
|
URI pointing to the JSON template for rendering |
|
|
delimiter used for separating rendered |
|
|
append |
|
|
truncate string values longer than the specified limit (defaults to 16384 set
by |
|
|
suffix to append to strings truncated due to exceeding |
|
|
recycling strategy that can either be |
Additional event template fields
Additional event template fields are a convenient short-cut to add custom fields
to a template or override the existing ones. Following configuration overrides
the host field of the GelfLayout.json template and adds two new custom
fields:
<JsonTemplateLayout eventTemplateUri="classpath:GelfLayout.json">
<EventTemplateAdditionalField key="host" value="www.apache.org"/>
<EventTemplateAdditionalField key="_serviceName" value="auth-service"/>
<EventTemplateAdditionalField key="_containerId" value="6ede3f0ca7d9"/>
</JsonTemplateLayout>
The default format for the added new fields are String.
One can also provide JSON-formatted additional fields:
<JsonTemplateLayout eventTemplateUri="classpath:GelfLayout.json">
<EventTemplateAdditionalField
key="marker"
format="JSON"
value='{"$resolver": "marker", "field": "name"}'/>
<EventTemplateAdditionalField
key="aNumber"
format="JSON"
value="1"/>
<EventTemplateAdditionalField
key="aList"
format="JSON"
value='[1, 2, "three"]'/>
</JsonTemplateLayout>
Additional event template fields can very well be introduced using properties-, YAML-, and JSON-formatted configurations:
appender.console.layout.type = JsonTemplateLayout
appender.console.layout.eventTemplateUri = classpath:GelfLayout.json
appender.console.layout.eventTemplateAdditionalField[0].type = EventTemplateAdditionalField
appender.console.layout.eventTemplateAdditionalField[0].key = marker
appender.console.layout.eventTemplateAdditionalField[0].value = {"$resolver": "marker", "field": "name"}
appender.console.layout.eventTemplateAdditionalField[0].format = JSON
appender.console.layout.eventTemplateAdditionalField[1].type = EventTemplateAdditionalField
appender.console.layout.eventTemplateAdditionalField[1].key = aNumber
appender.console.layout.eventTemplateAdditionalField[1].value = 1
appender.console.layout.eventTemplateAdditionalField[1].format = JSON
appender.console.layout.eventTemplateAdditionalField[2].type = EventTemplateAdditionalField
appender.console.layout.eventTemplateAdditionalField[2].key = aList
appender.console.layout.eventTemplateAdditionalField[2].value = [1, 2, "three"]
appender.console.layout.eventTemplateAdditionalField[2].format = JSON
JsonTemplateLayout:
eventTemplateAdditionalField:
- key: "marker"
value: '{"$resolver": "marker", "field": "name"}'
format: "JSON"
- key: "aNumber"
value: "1"
format: "JSON"
- key: "aList"
value: '[1, 2, "three"]'
format: "JSON"
{
"JsonTemplateLayout": {
"eventTemplateAdditionalField": [
{
"key": "marker",
"value": "{\"$resolver\": \"marker\", \"field\": \"name\"}",
"format": "JSON"
},
{
"key": "aNumber",
"value": "1",
"format": "JSON"
},
{
"key": "aList",
"value": "[1, 2, \"three\"]",
"format": "JSON"
}
]
}
}
Recycling strategy
RecyclerFactory plays a crucial role for determining the memory footprint of
the layout. Template resolvers employ it to create recyclers for objects that
they can reuse. The behavior of each RecyclerFactory and when one should
prefer one over another is explained below:
-
dummyperforms no recycling, hence each recycling attempt will result in a new instance. This will obviously create a load on the garbage-collector. It is a good choice for applications with low and medium log rate. -
threadLocalperforms the best, since every instance is stored inThreadLocals and accessed without any synchronization cost. Though this might not be a desirable option for applications running with hundreds of threads or more, e.g., a web servlet. -
queueis the best of both worlds. It allows recycling of objects up to a certain number (capacity). When this limit is exceeded due to excessive concurrent load (e.g.,capacityis 50 but there are 51 threads concurrently trying to log), it starts allocating.queueis a good strategy wherethreadLocalis not desirable.queuealso accepts optionalsupplier(of typejava.util.Queue, defaults toorg.jctools.queues.MpmcArrayQueue.newif JCTools is in the classpath; otherwisejava.util.concurrent.ArrayBlockingQueue.new) andcapacity(of typeint, defaults tomax(8,2*cpuCount+1)) parameters:Example configurations ofqueuerecycling strategyqueue:supplier=org.jctools.queues.MpmcArrayQueue.new queue:capacity=10 queue:supplier=java.util.concurrent.ArrayBlockingQueue.new,capacity=50
The default RecyclerFactory is threadLocal, if
log4j2.enable.threadlocals=true; otherwise, queue.
See Extending Recycler Factories for details on how to introduce custom
RecyclerFactory implementations.
Template Configuration
Templates are configured by means of the following JsonTemplateLayout
parameters:
-
eventTemplate[Uri](for serializingLogEvents) -
stackTraceElementTemplate[Uri](for serializingStackStraceElements) -
eventTemplateAdditionalField(for extending the used event template)
Event Templates
eventTemplate[Uri] describes the JSON structure JsonTemplateLayout uses to
serialize LogEvents. The default configuration (accessible by
log4j.layout.jsonTemplate.eventTemplate[Uri] property) is set to
classpath:EcsLayout.json provided by the log4j-layout-template-json
artifact, which contains the following predefined event templates:
-
EcsLayout.jsondescribed by the Elastic Common Schema (ECS) specification -
LogstashJsonEventLayoutV1.jsondescribed in Logstashjson_eventpattern for log4j -
GelfLayout.jsondescribed by the Graylog Extended Log Format (GELF) payload specification with additional_threadand_loggerfields. (Here it is advised to override the obligatoryhostfield with a user provided constant via additional event template fields to avoidhostNameproperty lookup at runtime, which incurs an extra cost.) -
GcpLayout.jsondescribed by Google Cloud Platform structured logging with additional_thread,_loggerand_exceptionfields. The exception trace, if any, is written to the_exceptionfield as well as themessagefield – the former is useful for explicitly searching/analyzing structured exception information, while the latter is Google’s expected place for the exception, and integrates with Google Error Reporting. -
JsonLayout.jsonproviding the exact JSON structure generated byJsonLayoutwith the exception ofthrownfield. (JsonLayoutserializes theThrowableas is via JacksonObjectMapper, whereasJsonLayout.jsontemplate ofJsonTemplateLayoutemploys theStackTraceElementLayout.jsontemplate for stack traces to generate a document-store-friendly flat structure.)
Event Template Resolvers
Event template resolvers consume a LogEvent and render a certain property of
it at the point of the JSON where they are declared. For instance, marker
resolver renders the marker of the event, level resolver renders the level,
and so on. An event template resolver is denoted with a special object
containing a`$resolver` key:
level resolver{
"version": "1.0",
"level": {
"$resolver": "level",
"field": "name"
}
}
Here version field will be rendered as is, while level field will be
populated by the level resolver. That is, this template will generate JSON
similar to the following:
{
"version": "1.0",
"level": "INFO"
}
The complete list of available event template resolvers are provided below in detail.
counter
config = [ start ] , [ overflowing ] , [ stringified ]
start = "start" -> number
overflowing = "overflowing" -> boolean
stringified = "stringified" -> boolean
Resolves a number from an internal counter.
Unless provided, start and overflowing are respectively set to zero and
true by default.
When stringified is enabled, which is set to `false by default, the resolved
number will be converted to a string.
|
Warning
|
When |
Examples
Resolves a sequence of numbers starting from 0. Once Long.MAX_VALUE is
reached, counter overflows to Long.MIN_VALUE.
{
"$resolver": "counter"
}
Resolves a sequence of numbers starting from 1000. Once Long.MAX_VALUE is
reached, counter overflows to Long.MIN_VALUE.
{
"$resolver": "counter",
"start": 1000
}
Resolves a sequence of numbers starting from 0 and keeps on doing as long as JVM heap allows.
{
"$resolver": "counter",
"overflowing": false
}
caseConverter
config = case , input , [ locale ] , [ errorHandlingStrategy ]
input = JSON
case = "case" -> ( "upper" | "lower" )
locale = "locale" -> (
language |
( language , "_" , country ) |
( language , "_" , country , "_" , variant )
)
errorHandlingStrategy = "errorHandlingStrategy" -> (
"fail" |
"pass" |
"replace"
)
replacement = "replacement" -> JSON
Converts the case of string values.
input can be any available template value; e.g., a JSON literal, a lookup
string, an object pointing to another resolver.
Unless provided, locale points to the one returned by
JsonTemplateLayoutDefaults.getLocale(), which is configured by
log4j.layout.jsonTemplate.locale system property and by default set to the
default system locale.
errorHandlingStrategy determines the behavior when either the input doesn’t
resolve to a string value or case conversion throws an exception:
-
failpropagates the failure -
passcauses the resolved value to be passed as is -
replacesuppresses the failure and replaces it with thereplacement, which is set tonullby default
errorHandlingStrategy is set to replace by default.
Most of the time JSON logs are persisted to a storage solution (e.g.,
Elasticsearch) that keeps a statically-typed index on fields. Hence, if a field
is always expected to be of type string, using non-string replacements or
pass in errorHandlingStrategy might result in type incompatibility issues at
the storage level.
|
Warning
|
Unless the input value is |
Examples
Convert the resolved log level strings to upper-case:
{
"$resolver": "caseConverter",
"case": "upper",
"input": {
"$resolver": "level",
"field": "name"
}
}
Convert the resolved USER environment variable to lower-case using nl_NL
locale:
{
"$resolver": "caseConverter",
"case": "lower",
"locale": "nl_NL",
"input": "${env:USER}"
}
Convert the resolved sessionId thread context data (MDC) to lower-case:
{
"$resolver": "caseConverter",
"case": "lower",
"input": {
"$resolver": "mdc",
"key": "sessionId"
}
}
Above, if sessionId MDC resolves to a, say, number, case conversion will fail.
Since errorHandlingStrategy is set to replace and replacement is set to
null by default, the resolved value will be null. One can suppress this
behavior and let the resolved sessionId number be left as is:
{
"$resolver": "caseConverter",
"case": "lower",
"input": {
"$resolver": "mdc",
"key": "sessionId"
},
"errorHandlingStrategy": "pass"
}
or replace it with a custom string:
{
"$resolver": "caseConverter",
"case": "lower",
"input": {
"$resolver": "mdc",
"key": "sessionId"
},
"errorHandlingStrategy": "replace",
"replacement": "unknown"
}
endOfBatch
{
"$resolver": "endOfBatch"
}
Resolves logEvent.isEndOfBatch() boolean flag.
exception
config = field , [ stringified ] , [ stackTrace ]
field = "field" -> ( "className" | "message" | "stackTrace" )
stackTrace = "stackTrace" -> (
[ stringified ]
, [ elementTemplate ]
)
stringified = "stringified" -> ( boolean | truncation )
truncation = "truncation" -> (
[ suffix ]
, [ pointMatcherStrings ]
, [ pointMatcherRegexes ]
)
suffix = "suffix" -> string
pointMatcherStrings = "pointMatcherStrings" -> string[]
pointMatcherRegexes = "pointMatcherRegexes" -> string[]
elementTemplate = "elementTemplate" -> object
Resolves fields of the Throwable returned by logEvent.getThrown().
stringified is set to false by default. stringified at the root level is
deprecated in favor of stackTrace.stringified, which has precedence if both
are provided.
pointMatcherStrings and pointMatcherRegexes enable the truncation of
stringified stack traces after the given matching point. If both parameters are
provided, pointMatcherStrings will be checked first.
If a stringified stack trace truncation takes place, it will be indicated with a
suffix, which by default is set to the configured truncatedStringSuffix in
the layout, unless explicitly provided. Every truncation suffix is prefixed with
a newline.
Stringified stack trace truncation operates in Caused by: and Suppressed:
label blocks. That is, matchers are executed against each label in isolation.
elementTemplate is an object describing the template to be used while
resolving the StackTraceElement array. If stringified is set to true,
elementTemplate will be discarded. By default, elementTemplate is set to
null and rather populated from the layout configuration. That is, the stack
trace element template can also be provided using
stackTraceElementTemplate[Uri] layout configuration parameters. The template
to be employed is determined in the following order:
-
elementTemplateprovided in the resolver configuration -
stackTraceElementTemplateparameter from layout configuration (the default is populated fromlog4j.layout.jsonTemplate.stackTraceElementTemplatesystem property) -
stackTraceElementTemplateUriparameter from layout configuration (the default is populated fromlog4j.layout.jsonTemplate.stackTraceElementTemplateUrisystem property)
See Stack Trace Element Templates for the list of available resolvers in a stack trace element template.
Note that this resolver is toggled by
log4j.layout.jsonTemplate.stackTraceEnabled property.
|
Warning
|
Since Each |
Examples
Resolve logEvent.getThrown().getClass().getCanonicalName():
{
"$resolver": "exception",
"field": "className"
}
Resolve the stack trace into a list of StackTraceElement objects:
{
"$resolver": "exception",
"field": "stackTrace"
}
Resolve the stack trace into a string field:
{
"$resolver": "exception",
"field": "stackTrace",
"stackTrace": {
"stringified": true
}
}
Resolve the stack trace into a string field such that the content will be truncated after the given point matcher:
{
"$resolver": "exception",
"field": "stackTrace",
"stackTrace": {
"stringified": {
"truncation": {
"suffix": "... [truncated]",
"pointMatcherStrings": ["at javax.servlet.http.HttpServlet.service"]
}
}
}
}
Resolve the stack trace into an object described by the provided stack trace element template:
{
"$resolver": "exception",
"field": "stackTrace",
"stackTrace": {
"elementTemplate": {
"class": {
"$resolver": "stackTraceElement",
"field": "className"
},
"method": {
"$resolver": "stackTraceElement",
"field": "methodName"
},
"file": {
"$resolver": "stackTraceElement",
"field": "fileName"
},
"line": {
"$resolver": "stackTraceElement",
"field": "lineNumber"
}
}
}
}
See Stack Trace Element Templates for further details on resolvers available
for StackTraceElement templates.
exceptionRootCause
Resolves the fields of the innermost Throwable returned by
logEvent.getThrown(). Its syntax and garbage-footprint are identical to the
exception resolver.
level
config = field , [ severity ]
field = "field" -> ( "name" | "severity" )
severity = severity-field
severity-field = "field" -> ( "keyword" | "code" )
Resolves the fields of the logEvent.getLevel().
Examples
Resolve the level name:
{
"$resolver": "level",
"field": "name"
}
Resolve the Syslog severity keyword:
{
"$resolver": "level",
"field": "severity",
"severity": {
"field": "keyword"
}
}
Resolve the Syslog severity code:
{
"$resolver": "level",
"field": "severity",
"severity": {
"field": "code"
}
}
logger
config = "field" -> ( "name" | "fqcn" )
Resolves logEvent.getLoggerFqcn() and logEvent.getLoggerName().
Examples
Resolve the logger name:
{
"$resolver": "logger",
"field": "name"
}
Resolve the logger’s fully qualified class name:
{
"$resolver": "logger",
"field": "fqcn"
}
main
config = ( index | key )
index = "index" -> number
key = "key" -> string
Performs Main Argument Lookup for the
given index or key.
Examples
Resolve the 1st main() method argument:
{
"$resolver": "main",
"index": 0
}
Resolve the argument coming right after --userId:
{
"$resolver": "main",
"key": "--userId"
}
map
Resolves MapMessages. See Map Resolver Template
for details.
marker
config = "field" -> ( "name" | "parents" )
Resolves logEvent.getMarker().
Examples
Resolve the marker name:
{
"$resolver": "marker",
"field": "name"
}
Resolve the names of the marker’s parents:
{
"$resolver": "marker",
"field": "parents"
}
mdc
Resolves Mapped Diagnostic Context (MDC), aka. Thread Context Data. See Map Resolver Template for details.
|
Warning
|
|
message
config = [ stringified ] , [ fallbackKey ]
stringified = "stringified" -> boolean
fallbackKey = "fallbackKey" -> string
Resolves logEvent.getMessage().
|
Warning
|
For simple string messages, the resolution is performed without allocations.
For |
Examples
Resolve the message into a string:
{
"$resolver": "message",
"stringified": true
}
Resolve the message such that if it is an ObjectMessage or a
MultiformatMessage with JSON support, its type (string, list, object, etc.)
will be retained:
{
"$resolver": "message"
}
Given the above configuration, a SimpleMessage will generate a "sample log
message", whereas a MapMessage will generate a {"action": "login",
"sessionId": "87asd97a"}. Certain indexed log storage systems (e.g.,
Elasticsearch) will not allow both values
to coexist due to type mismatch: one is a string while the other is an object.
Here one can use a fallbackKey to work around the problem:
{
"$resolver": "message",
"fallbackKey": "formattedMessage"
}
Using this configuration, a SimpleMessage will generate a
{"formattedMessage": "sample log message"} and a MapMessage will generate a
{"action": "login", "sessionId": "87asd97a"}. Note that both emitted JSONs are
of type object and have no type-conflicting fields.
messageParameter
config = [ stringified ] , [ index ]
stringified = "stringified" -> boolean
index = "index" -> number
Resolves logEvent.getMessage().getParameters().
|
Warning
|
Regarding garbage footprint, |
Examples
Resolve the message parameters into an array:
{
"$resolver": "messageParameter"
}
Resolve the string representation of all message parameters into an array:
{
"$resolver": "messageParameter",
"stringified": true
}
Resolve the first message parameter:
{
"$resolver": "messageParameter",
"index": 0
}
Resolve the string representation of the first message parameter:
{
"$resolver": "messageParameter",
"index": 0,
"stringified": true
}
ndc
config = [ pattern ]
pattern = "pattern" -> string
Resolves the Nested Diagnostic Context (NDC), aka. Thread Context Stack,
String[] returned by logEvent.getContextStack().
Examples
Resolve all NDC values into a list:
{
"$resolver": "ndc"
}
Resolve all NDC values matching with the pattern regex:
{
"$resolver": "ndc",
"pattern": "user(Role|Rank):\\w+"
}
pattern
config = pattern , [ stackTraceEnabled ]
pattern = "pattern" -> string
stackTraceEnabled = "stackTraceEnabled" -> boolean
Resolver delegating to PatternLayout.
The default value of stackTraceEnabled is inherited from the parent
JsonTemplateLayout.
Examples
Resolve the string produced by %p %c{1.} [%t] %X{userId} %X %m%ex pattern:
{
"$resolver": "pattern",
"pattern": "%p %c{1.} [%t] %X{userId} %X %m%ex"
}
source
config = "field" -> (
"className" |
"fileName" |
"methodName" |
"lineNumber" )
Resolves the fields of the StackTraceElement returned by
logEvent.getSource().
Note that this resolver is toggled by
log4j.layout.jsonTemplate.locationInfoEnabled property.
Examples
Resolve the line number:
{
"$resolver": "source",
"field": "lineNumber"
}
thread
config = "field" -> ( "name" | "id" | "priority" )
Resolves logEvent.getThreadId(), logEvent.getThreadName(),
logEvent.getThreadPriority().
Examples
Resolve the thread name:
{
"$resolver": "thread",
"field": "name"
}
timestamp
config = [ patternConfig | epochConfig ]
patternConfig = "pattern" -> ( [ format ] , [ timeZone ] , [ locale ] )
format = "format" -> string
timeZone = "timeZone" -> string
locale = "locale" -> (
language |
( language , "_" , country ) |
( language , "_" , country , "_" , variant )
)
epochConfig = "epoch" -> ( unit , [ rounded ] )
unit = "unit" -> (
"nanos" |
"millis" |
"secs" |
"millis.nanos" |
"secs.nanos" |
)
rounded = "rounded" -> boolean
Resolves logEvent.getInstant() in various forms.
Examples
Configuration |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Map Resolver Template
ReadOnlyStringMap is Log4j’s Map<String, Object> equivalent with
garbage-free accessors and heavily employed throughout the code base. It is the
data structure backing both Mapped Diagnostic Context (MDC), aka. Thread Context
Data and MapMessage implementations. Hence template resolvers for both of
these are provided by a single backend: ReadOnlyStringMapResolver. Put another
way, both mdc and map resolvers support identical configuration, behaviour,
and garbage footprint, which are detailed below.
config = singleAccess | multiAccess
singleAccess = key , [ stringified ]
key = "key" -> string
stringified = "stringified" -> boolean
multiAccess = [ pattern ] , [ replacement ] , [ flatten ] , [ stringified ]
pattern = "pattern" -> string
replacement = "replacement" -> string
flatten = "flatten" -> ( boolean | flattenConfig )
flattenConfig = [ flattenPrefix ]
flattenPrefix = "prefix" -> string
singleAccess resolves a single field, whilst multiAccess resolves a
multitude of fields. If flatten is provided, multiAccess merges the fields
with the parent, otherwise creates a new JSON object containing the values.
Enabling stringified flag converts each value to its string representation.
Regex provided in the pattern is used to match against the keys. If provided,
replacement will be used to replace the matched keys. These two are
effectively equivalent to Pattern.compile(pattern).matcher(key).matches() and
Pattern.compile(pattern).matcher(key).replaceAll(replacement) calls.
|
Warning
|
Regarding garbage footprint,
Writing certain non-primitive values (e.g., |
"$resolver" is left out in the following examples, since it is to be
defined by the actual resolver, e.g., map, mdc.
Resolve the value of the field keyed with user:role:
{
"$resolver": "…",
"key": "user:role"
}
Resolve the string representation of the user:rank field value:
{
"$resolver": "…",
"key": "user:rank",
"stringified": true
}
Resolve all fields into an object:
{
"$resolver": "…"
}
Resolve all fields into an object such that values are converted to string:
{
"$resolver": "…",
"stringified": true
}
Resolve all fields whose keys match with the user:(role|rank) regex into an
object:
{
"$resolver": "…",
"pattern": "user:(role|rank)"
}
Resolve all fields whose keys match with the user:(role|rank) regex into an
object after removing the user: prefix in the key:
{
"$resolver": "…",
"pattern": "user:(role|rank)",
"replacement": "$1"
}
Merge all fields whose keys are matching with the user:(role|rank) regex into
the parent:
{
"$resolver": "…",
"flatten": true,
"pattern": "user:(role|rank)"
}
After converting the corresponding field values to string, merge all fields to
parent such that keys are prefixed with _:
{
"$resolver": "…",
"stringified": true,
"flatten": {
"prefix": "_"
}
}
Stack Trace Element Templates
exception and
exceptionRootCause event template resolvers can
serialize an exception stack trace (i.e., StackTraceElement[] returned by
Throwable#getStackTrace()) into a JSON array. While doing so, JSON templating
infrastructure is used again.
stackTraceElement[Uri] describes the JSON structure JsonTemplateLayout uses
to format StackTraceElements. The default configuration (accessible by
log4j.layout.jsonTemplate.stackTraceElementTemplate[Uri] property) is set to
classpath:StackTraceElementLayout.json provided by the
log4j-layout-template-json artifact:
{
"class": {
"$resolver": "stackTraceElement",
"field": "className"
},
"method": {
"$resolver": "stackTraceElement",
"field": "methodName"
},
"file": {
"$resolver": "stackTraceElement",
"field": "fileName"
},
"line": {
"$resolver": "stackTraceElement",
"field": "lineNumber"
}
}
The allowed template configuration syntax is as follows:
config = "field" -> (
"className" |
"fileName" |
"methodName" |
"lineNumber" )
All above accesses to StackTraceElement is garbage-free.
Extending
JsonTemplateLayout relies on Log4j plugin system to build
up the features it provides. This enables feature customization a breeze for
users. As of this moment, following features are implemented by means of
plugins:
-
Event template resolvers (e.g.,
exception,message,levelevent template resolvers) -
Event template interceptors (e.g., injection of
eventTemplateAdditionalField) -
Recycler factories
Following sections cover these in detail.
Plugin Preliminaries
Log4j plugin system is the de facto extension mechanism embraced by various
Log4j components, including JsonTemplateLayout. Plugins make it possible
for extensible components receive feature implementations without any explicit
links in between. It is analogous to a
dependency injection
framework, but curated for Log4j-specific needs.
In a nutshell, you annotate your classes with @Plugin and their (static)
creator methods with @PluginFactory. Last, you inform the Log4j plugin system
to discover these custom classes. This can be done either using packages
declared in your Log4j configuration or by various other ways described in
plugin system documentation.
Extending Event Resolvers
All available event template resolvers are simple
plugins employed by JsonTemplateLayout. To add new ones, one just needs to
create their own EventResolver and instruct its injection via a
@Plugin-annotated EventResolverFactory class.
For demonstration purposes, below we will create a randomNumber event resolver.
Let’s start with the actual resolver:
package com.acme.logging.log4j.layout.template.json;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolver;
import org.apache.logging.log4j.layout.template.json.util.JsonWriter;
/**
* Resolves a random floating point number.
*
* <h3>Configuration</h3>
*
* <pre>
* config = ( [ range ] )
* range = number[]
* </pre>
*
* {@code range} is a number array with two elements, where the first number
* denotes the start (inclusive) and the second denotes the end (exclusive).
* {@code range} is optional and by default set to {@code [0, 1]}.
*
* <h3>Examples</h3>
*
* Resolve a random number between 0 and 1:
*
* <pre>
* {
* "$resolver": "randomNumber"
* }
* </pre>
*
* Resolve a random number between -0.123 and 0.123:
*
* <pre>
* {
* "$resolver": "randomNumber",
* "range": [-0.123, 0.123]
* }
* </pre>
*/
public final class RandomNumberResolver implements EventResolver {
private final double loIncLimit;
private final double hiExcLimit;
RandomNumberResolver(final TemplateResolverConfig config) {
final List<Number> rangeArray = config.getList("range", Number.class);
if (rangeArray == null) {
this.loIncLimit = 0D;
this.hiExcLimit = 1D;
} else if (rangeArray.size() != 2) {
throw new IllegalArgumentException(
"range array must be of size two: " + config);
} else {
this.loIncLimit = rangeArray.get(0).doubleValue();
this.hiExcLimit = rangeArray.get(1).doubleValue();
if (loIncLimit > hiExcLimit) {
throw new IllegalArgumentException("invalid range: " + config);
}
}
}
static String getName() {
return "randomNumber";
}
@Override
public void resolve(
final LogEvent value,
final JsonWriter jsonWriter) {
final double randomNumber =
loIncLimit + (hiExcLimit - loIncLimit) * Math.random();
jsonWriter.writeNumber(randomNumber);
}
}
Next create a EventResolverFactory class to register RandomNumberResolver
into the Log4j plugin system.
RandomNumberResolver into the Log4j plugin systempackage com.acme.logging.log4j.layout.template.json;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.config.plugins.PluginFactory;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverContext;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverFactory;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolver;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverConfig;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverFactory;
/**
* {@link RandomNumberResolver} factory.
*/
@Plugin(name = "RandomNumberResolverFactory", category = TemplateResolverFactory.CATEGORY)
public final class RandomNumberResolverFactory implements EventResolverFactory {
private static final RandomNumberResolverFactory INSTANCE =
new RandomNumberResolverFactory();
private RandomNumberResolverFactory() {}
@PluginFactory
public static RandomNumberResolverFactory getInstance() {
return INSTANCE;
}
@Override
public String getName() {
return RandomNumberResolver.getName();
}
@Override
public RandomNumberResolver create(
final EventResolverContext context,
final TemplateResolverConfig config) {
return new RandomNumberResolver(config);
}
}
Almost complete. Last, we need to inform the Log4j plugin system to discover these custom classes:
randomNumber resolver<?xml version="1.0" encoding="UTF-8"?>
<Configuration packages="com.acme.logging.log4j.layout.template.json">
<!-- ... -->
<JsonTemplateLayout>
<EventTemplateAdditionalField
key="id"
format="JSON"
value='{"$resolver": "randomNumber", "range": [0, 1000000]}'/>
</JsonTemplateLayout>
<!-- ... -->
</Configuration>
All available event template resolvers are located in
org.apache.logging.log4j.layout.template.json.resolver package. It is a fairly
rich resource for inspiration while implementing new resolvers.
Intercepting the Template Resolver Compiler
JsonTemplateLayout allows interception of the template resolver compilation,
which is the process converting a template into a Java function performing the
JSON serialization. This interception mechanism is internally used to implement
eventTemplateRootObjectKey and eventTemplateAdditionalField features. In a
nutshell, one needs to create a @Plugin-annotated class extending from
EventResolverInterceptor interface.
To see the interception in action, check out the EventRootObjectKeyInterceptor
class which is responsible for implementing the eventTemplateRootObjectKey
feature:
eventTemplateRootObjectKey, if presentimport org.apache.logging.log4j.layout.template.json.resolver.EventResolverContext;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverInterceptor;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverInterceptor;
/**
* Interceptor to add a root object key to the event template.
*/
@Plugin(name = "EventRootObjectKeyInterceptor", category = TemplateResolverInterceptor.CATEGORY)
public class EventRootObjectKeyInterceptor implements EventResolverInterceptor {
private static final EventRootObjectKeyInterceptor INSTANCE =
new EventRootObjectKeyInterceptor();
private EventRootObjectKeyInterceptor() {}
@PluginFactory
public static EventRootObjectKeyInterceptor getInstance() {
return INSTANCE;
}
@Override
public Object processTemplateBeforeResolverInjection(
final EventResolverContext context,
final Object node) {
String eventTemplateRootObjectKey = context.getEventTemplateRootObjectKey();
return eventTemplateRootObjectKey != null
? Collections.singletonMap(eventTemplateRootObjectKey, node)
: node;
}
}
Here, processTemplateBeforeResolverInjection() method checks if the user has
provided an eventTemplateRootObjectKey. If so, it wraps the root node with a
new object; otherwise, returns the node as is. Note that node refers to the
root Java object of the event template read by JsonReader.
Extending Recycler Factories
recyclerFactory input String read from the layout configuration is converted
to a RecyclerFactory using the default RecyclerFactoryConverter extending
from TypeConverter<RecyclerFactory>. If one wants to change this behavior,
they simply need to add their own TypeConverter<RecyclerFactory> implementing
Comparable<TypeConverter<?>> to prioritize their custom converter.
TypeConverter for RecyclerFactorypackage com.acme.logging.log4j.layout.template.json;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.config.plugins.convert.TypeConverter;
import org.apache.logging.log4j.core.config.plugins.convert.TypeConverters;
@Plugin(name = "AcmeRecyclerFactoryConverter", category = TypeConverters.CATEGORY)
public final class AcmeRecyclerFactoryConverter
implements TypeConverter<RecyclerFactory>, Comparable<TypeConverter<?>> {
@Override
public RecyclerFactory convert(final String recyclerFactorySpec) {
return AcmeRecyclerFactory.ofSpec(recyclerFactorySpec);
}
@Override
public int compareTo(final TypeConverter<?> ignored) {
return -1;
}
}
Here note that compareTo() always returns -1 to rank it higher compared to
other matching converters.
Features
Below is a feature comparison matrix between JsonTemplateLayout and
alternatives.
Feature |
|
|||
Java version |
8 |
8 |
8 |
6 |
Dependencies |
None |
Jackson |
None |
None |
Schema customization? |
✓ |
✕ |
✕ |
✕ |
Timestamp customization? |
✓ |
✕ |
✕ |
✕ |
(Almost) garbage-free? |
✓ |
✕ |
✓ |
✓ |
Custom typed |
✓ |
✕ |
✕ |
?[1] |
Custom typed |
✓ |
✕ |
✕ |
✕ |
Rendering stack traces as array? |
✓ |
✓ |
✕ |
✓ |
Stack trace truncation? |
✓ |
✕ |
✕ |
✕ |
JSON pretty print? |
✕ |
✓ |
✕ |
✕ |
Additional string fields? |
✓ |
✓ |
✓ |
✓ |
Additional JSON fields? |
✓ |
✕ |
✕ |
✕ |
Custom resolvers? |
✓ |
✕ |
✕ |
✕ |
F.A.Q.
Are lookups supported in templates?
Yes, lookups (e.g., ${java:version},
${env:USER}, ${date:MM-dd-yyyy}) are supported in string
literals of templates. Though note that they are not garbage-free.
Are recursive collections supported?
No. Consider a Message containing a recursive value as follows:
Object[] recursiveCollection = new Object[1];
recursiveCollection[0] = recursiveCollection;
While the exact exception might vary, you will most like get a
StackOverflowError for trying to render recursiveCollection into a
String. Note that this is also the default behaviour for other Java standard
library methods, e.g., Arrays.toString(). Hence mind self references while
logging.
Is JsonTemplateLayout garbage-free?
Yes, if the garbage-free layout behaviour toggling properties
log4j2.enableDirectEncoders and log4j2.garbagefreeThreadContextMap are
enabled. Take into account the following caveats:
-
The configured recycling strategy might not be garbage-free.
-
Since
Throwable#getStackTrace()clones the originalStackTraceElement[], access to (and hence rendering of) stack traces are not garbage-free. -
Serialization of
MapMessages andObjectMessages are mostly garbage-free except for certain types (e.g.,BigDecimal,BigInteger,Collections, exceptList). -
Lookups (that is,
${…}variables) are not garbage-free.
Don’t forget to check out the notes on garbage footprint of resolvers you employ in templates.
ObjectMessages and if Jackson is in the classpath.


