<dependency>
<groupId>org.apache.logging.log4j</groupId>
<artifactId>log4j-layout-template-json</artifactId>
<version>2.21.0</version>
</dependency>
JSON Template Layout
JsonTemplateLayout
is a customizable, efficient, and garbage-free JSON
generating layout. It encodes LogEvent
s according to the structure described
by the JSON template provided. In a nutshell, it shines with its
-
Customizable JSON structure (see
eventTemplate[Uri]
andstackTraceElementTemplate[Uri]
layout configuration parameters) -
Customizable timestamp formatting (see
timestamp
event template resolver) -
Feature rich exception formatting (see
exception
andexceptionRootCause
event template resolvers) -
Customizable object recycling strategy
Usage
Adding log4j-layout-template-json
artifact to your list of dependencies is
enough to enable access to JsonTemplateLayout
in your Log4j configuration:
For instance, given the following JSON template modelling
the Elastic Common Schema (ECS) specification
(accessible via classpath:EcsLayout.json
)
{
"@timestamp": {
"$resolver": "timestamp",
"pattern": {
"format": "yyyy-MM-dd'T'HH:mm:ss.SSS'Z'",
"timeZone": "UTC"
}
},
"ecs.version": "1.2.0",
"log.level": {
"$resolver": "level",
"field": "name"
},
"message": {
"$resolver": "message",
"stringified": true
},
"process.thread.name": {
"$resolver": "thread",
"field": "name"
},
"log.logger": {
"$resolver": "logger",
"field": "name"
},
"labels": {
"$resolver": "mdc",
"flatten": true,
"stringified": true
},
"tags": {
"$resolver": "ndc"
},
"error.type": {
"$resolver": "exception",
"field": "className"
},
"error.message": {
"$resolver": "exception",
"field": "message"
},
"error.stack_trace": {
"$resolver": "exception",
"field": "stackTrace",
"stackTrace": {
"stringified": true
}
}
}
in combination with the below log4j2.xml
configuration:
<JsonTemplateLayout eventTemplateUri="classpath:EcsLayout.json"/>
or with the below log4j2.properties
configuration:
appender.console.layout.type = JsonTemplateLayout
appender.console.layout.eventTemplateUri = classpath:EcsLayout.json
JsonTemplateLayout
generates JSON as follows:
{
"@timestamp": "2017-05-25T19:56:23.370Z",
"ecs.version": "1.2.0",
"log.level": "ERROR",
"message": "Hello, error!",
"process.thread.name": "main",
"log.logger": "org.apache.logging.log4j.JsonTemplateLayoutDemo",
"error.type": "java.lang.RuntimeException",
"error.message": "test",
"error.stack_trace": "java.lang.RuntimeException: test\n\tat org.apache.logging.log4j.JsonTemplateLayoutDemo.main(JsonTemplateLayoutDemo.java:11)\n"
}
Layout Configuration
JsonTemplateLayout
is configured with the following parameters:
|
|
Description |
|
|
|
|
|
toggles access to the |
|
|
toggles access to the stack traces (defaults to |
|
|
inline JSON template for rendering |
|
|
URI pointing to the JSON template for rendering |
|
|
if present, the event template is put into a JSON object composed of a single
member with the provided key (defaults to |
|
|
additional key-value pairs appended to the root of the event template |
|
|
inline JSON template for rendering |
|
|
URI pointing to the JSON template for rendering |
|
|
delimiter used for separating rendered |
|
|
append |
|
|
truncate string values longer than the specified limit (defaults to 16384 set
by |
|
|
suffix to append to strings truncated due to exceeding |
|
|
recycling strategy that can either be |
Additional event template fields
Additional event template fields are a convenient short-cut to add custom fields
to a template or override the existing ones. Following configuration overrides
the host
field of the GelfLayout.json
template and adds two new custom
fields:
<JsonTemplateLayout eventTemplateUri="classpath:GelfLayout.json">
<EventTemplateAdditionalField key="host" value="www.apache.org"/>
<EventTemplateAdditionalField key="_serviceName" value="auth-service"/>
<EventTemplateAdditionalField key="_containerId" value="6ede3f0ca7d9"/>
</JsonTemplateLayout>
The default format
for the added new fields are String
.
One can also provide JSON-formatted additional fields:
<JsonTemplateLayout eventTemplateUri="classpath:GelfLayout.json">
<EventTemplateAdditionalField
key="marker"
format="JSON"
value='{"$resolver": "marker", "field": "name"}'/>
<EventTemplateAdditionalField
key="aNumber"
format="JSON"
value="1"/>
<EventTemplateAdditionalField
key="aList"
format="JSON"
value='[1, 2, "three"]'/>
</JsonTemplateLayout>
Additional event template fields can very well be introduced using properties-, YAML-, and JSON-formatted configurations:
appender.console.layout.type = JsonTemplateLayout
appender.console.layout.eventTemplateUri = classpath:GelfLayout.json
appender.console.layout.eventTemplateAdditionalField[0].type = EventTemplateAdditionalField
appender.console.layout.eventTemplateAdditionalField[0].key = marker
appender.console.layout.eventTemplateAdditionalField[0].value = {"$resolver": "marker", "field": "name"}
appender.console.layout.eventTemplateAdditionalField[0].format = JSON
appender.console.layout.eventTemplateAdditionalField[1].type = EventTemplateAdditionalField
appender.console.layout.eventTemplateAdditionalField[1].key = aNumber
appender.console.layout.eventTemplateAdditionalField[1].value = 1
appender.console.layout.eventTemplateAdditionalField[1].format = JSON
appender.console.layout.eventTemplateAdditionalField[2].type = EventTemplateAdditionalField
appender.console.layout.eventTemplateAdditionalField[2].key = aList
appender.console.layout.eventTemplateAdditionalField[2].value = [1, 2, "three"]
appender.console.layout.eventTemplateAdditionalField[2].format = JSON
JsonTemplateLayout:
eventTemplateAdditionalField:
- key: "marker"
value: '{"$resolver": "marker", "field": "name"}'
format: "JSON"
- key: "aNumber"
value: "1"
format: "JSON"
- key: "aList"
value: '[1, 2, "three"]'
format: "JSON"
{
"JsonTemplateLayout": {
"eventTemplateAdditionalField": [
{
"key": "marker",
"value": "{\"$resolver\": \"marker\", \"field\": \"name\"}",
"format": "JSON"
},
{
"key": "aNumber",
"value": "1",
"format": "JSON"
},
{
"key": "aList",
"value": "[1, 2, \"three\"]",
"format": "JSON"
}
]
}
}
Recycling strategy
RecyclerFactory
plays a crucial role for determining the memory footprint of
the layout. Template resolvers employ it to create recyclers for objects that
they can reuse. The behavior of each RecyclerFactory
and when one should
prefer one over another is explained below:
-
dummy
performs no recycling, hence each recycling attempt will result in a new instance. This will obviously create a load on the garbage-collector. It is a good choice for applications with low and medium log rate. -
threadLocal
performs the best, since every instance is stored inThreadLocal
s and accessed without any synchronization cost. Though this might not be a desirable option for applications running with hundreds of threads or more, e.g., a web servlet. -
queue
is the best of both worlds. It allows recycling of objects up to a certain number (capacity
). When this limit is exceeded due to excessive concurrent load (e.g.,capacity
is 50 but there are 51 threads concurrently trying to log), it starts allocating.queue
is a good strategy wherethreadLocal
is not desirable.queue
also accepts optionalsupplier
(of typejava.util.Queue
, defaults toorg.jctools.queues.MpmcArrayQueue.new
if JCTools is in the classpath; otherwisejava.util.concurrent.ArrayBlockingQueue.new
) andcapacity
(of typeint
, defaults tomax(8,2*cpuCount+1)
) parameters:Example configurations ofqueue
recycling strategyqueue:supplier=org.jctools.queues.MpmcArrayQueue.new queue:capacity=10 queue:supplier=java.util.concurrent.ArrayBlockingQueue.new,capacity=50
The default RecyclerFactory
is threadLocal
, if
log4j2.enable.threadlocals=true
; otherwise, queue
.
See Extending Recycler Factories for details on how to introduce custom
RecyclerFactory
implementations.
Template Configuration
Templates are configured by means of the following JsonTemplateLayout
parameters:
-
eventTemplate[Uri]
(for serializingLogEvent
s) -
stackTraceElementTemplate[Uri]
(for serializingStackStraceElement
s) -
eventTemplateAdditionalField
(for extending the used event template)
Event Templates
eventTemplate[Uri]
describes the JSON structure JsonTemplateLayout
uses to
serialize LogEvent
s. The default configuration (accessible by
log4j.layout.jsonTemplate.eventTemplate[Uri]
property) is set to
classpath:EcsLayout.json
provided by the log4j-layout-template-json
artifact, which contains the following predefined event templates:
-
EcsLayout.json
described by the Elastic Common Schema (ECS) specification -
LogstashJsonEventLayoutV1.json
described in Logstashjson_event
pattern for log4j -
GelfLayout.json
described by the Graylog Extended Log Format (GELF) payload specification with additional_thread
and_logger
fields. (Here it is advised to override the obligatoryhost
field with a user provided constant via additional event template fields to avoidhostName
property lookup at runtime, which incurs an extra cost.) -
GcpLayout.json
described by Google Cloud Platform structured logging with additional_thread
,_logger
and_exception
fields. The exception trace, if any, is written to the_exception
field as well as themessage
field – the former is useful for explicitly searching/analyzing structured exception information, while the latter is Google’s expected place for the exception, and integrates with Google Error Reporting. -
JsonLayout.json
providing the exact JSON structure generated byJsonLayout
with the exception ofthrown
field. (JsonLayout
serializes theThrowable
as is via JacksonObjectMapper
, whereasJsonLayout.json
template ofJsonTemplateLayout
employs theStackTraceElementLayout.json
template for stack traces to generate a document-store-friendly flat structure.)
Event Template Resolvers
Event template resolvers consume a LogEvent
and render a certain property of
it at the point of the JSON where they are declared. For instance, marker
resolver renders the marker of the event, level
resolver renders the level,
and so on. An event template resolver is denoted with a special object
containing a`$resolver` key:
level
resolver{
"version": "1.0",
"level": {
"$resolver": "level",
"field": "name"
}
}
Here version
field will be rendered as is, while level
field will be
populated by the level
resolver. That is, this template will generate JSON
similar to the following:
{
"version": "1.0",
"level": "INFO"
}
The complete list of available event template resolvers are provided below in detail.
counter
config = [ start ] , [ overflowing ] , [ stringified ]
start = "start" -> number
overflowing = "overflowing" -> boolean
stringified = "stringified" -> boolean
Resolves a number from an internal counter.
Unless provided, start
and overflowing
are respectively set to zero and
true
by default.
When stringified
is enabled, which is set to `false by default, the resolved
number will be converted to a string.
Warning
|
When |
Examples
Resolves a sequence of numbers starting from 0. Once Long.MAX_VALUE
is
reached, counter overflows to Long.MIN_VALUE
.
{
"$resolver": "counter"
}
Resolves a sequence of numbers starting from 1000. Once Long.MAX_VALUE
is
reached, counter overflows to Long.MIN_VALUE
.
{
"$resolver": "counter",
"start": 1000
}
Resolves a sequence of numbers starting from 0 and keeps on doing as long as JVM heap allows.
{
"$resolver": "counter",
"overflowing": false
}
caseConverter
config = case , input , [ locale ] , [ errorHandlingStrategy ]
input = JSON
case = "case" -> ( "upper" | "lower" )
locale = "locale" -> (
language |
( language , "_" , country ) |
( language , "_" , country , "_" , variant )
)
errorHandlingStrategy = "errorHandlingStrategy" -> (
"fail" |
"pass" |
"replace"
)
replacement = "replacement" -> JSON
Converts the case of string values.
input
can be any available template value; e.g., a JSON literal, a lookup
string, an object pointing to another resolver.
Unless provided, locale
points to the one returned by
JsonTemplateLayoutDefaults.getLocale()
, which is configured by
log4j.layout.jsonTemplate.locale
system property and by default set to the
default system locale.
errorHandlingStrategy
determines the behavior when either the input doesn’t
resolve to a string value or case conversion throws an exception:
-
fail
propagates the failure -
pass
causes the resolved value to be passed as is -
replace
suppresses the failure and replaces it with thereplacement
, which is set tonull
by default
errorHandlingStrategy
is set to replace
by default.
Most of the time JSON logs are persisted to a storage solution (e.g.,
Elasticsearch) that keeps a statically-typed index on fields. Hence, if a field
is always expected to be of type string, using non-string replacement
s or
pass
in errorHandlingStrategy
might result in type incompatibility issues at
the storage level.
Warning
|
Unless the input value is |
Examples
Convert the resolved log level strings to upper-case:
{
"$resolver": "caseConverter",
"case": "upper",
"input": {
"$resolver": "level",
"field": "name"
}
}
Convert the resolved USER
environment variable to lower-case using nl_NL
locale:
{
"$resolver": "caseConverter",
"case": "lower",
"locale": "nl_NL",
"input": "${env:USER}"
}
Convert the resolved sessionId
thread context data (MDC) to lower-case:
{
"$resolver": "caseConverter",
"case": "lower",
"input": {
"$resolver": "mdc",
"key": "sessionId"
}
}
Above, if sessionId
MDC resolves to a, say, number, case conversion will fail.
Since errorHandlingStrategy
is set to replace
and replacement is set to
null
by default, the resolved value will be null
. One can suppress this
behavior and let the resolved sessionId
number be left as is:
{
"$resolver": "caseConverter",
"case": "lower",
"input": {
"$resolver": "mdc",
"key": "sessionId"
},
"errorHandlingStrategy": "pass"
}
or replace it with a custom string:
{
"$resolver": "caseConverter",
"case": "lower",
"input": {
"$resolver": "mdc",
"key": "sessionId"
},
"errorHandlingStrategy": "replace",
"replacement": "unknown"
}
endOfBatch
{
"$resolver": "endOfBatch"
}
Resolves logEvent.isEndOfBatch()
boolean flag.
exception
config = field , [ stringified ] , [ stackTrace ]
field = "field" -> ( "className" | "message" | "stackTrace" )
stackTrace = "stackTrace" -> (
[ stringified ]
, [ elementTemplate ]
)
stringified = "stringified" -> ( boolean | truncation )
truncation = "truncation" -> (
[ suffix ]
, [ pointMatcherStrings ]
, [ pointMatcherRegexes ]
)
suffix = "suffix" -> string
pointMatcherStrings = "pointMatcherStrings" -> string[]
pointMatcherRegexes = "pointMatcherRegexes" -> string[]
elementTemplate = "elementTemplate" -> object
Resolves fields of the Throwable
returned by logEvent.getThrown()
.
stringified
is set to false
by default. stringified
at the root level is
deprecated in favor of stackTrace.stringified
, which has precedence if both
are provided.
pointMatcherStrings
and pointMatcherRegexes
enable the truncation of
stringified stack traces after the given matching point. If both parameters are
provided, pointMatcherStrings
will be checked first.
If a stringified stack trace truncation takes place, it will be indicated with a
suffix
, which by default is set to the configured truncatedStringSuffix
in
the layout, unless explicitly provided. Every truncation suffix is prefixed with
a newline.
Stringified stack trace truncation operates in Caused by:
and Suppressed:
label blocks. That is, matchers are executed against each label in isolation.
elementTemplate
is an object describing the template to be used while
resolving the StackTraceElement
array. If stringified
is set to true
,
elementTemplate
will be discarded. By default, elementTemplate
is set to
null
and rather populated from the layout configuration. That is, the stack
trace element template can also be provided using
stackTraceElementTemplate[Uri]
layout configuration parameters. The template
to be employed is determined in the following order:
-
elementTemplate
provided in the resolver configuration -
stackTraceElementTemplate
parameter from layout configuration (the default is populated fromlog4j.layout.jsonTemplate.stackTraceElementTemplate
system property) -
stackTraceElementTemplateUri
parameter from layout configuration (the default is populated fromlog4j.layout.jsonTemplate.stackTraceElementTemplateUri
system property)
See Stack Trace Element Templates for the list of available resolvers in a stack trace element template.
Note that this resolver is toggled by
log4j.layout.jsonTemplate.stackTraceEnabled
property.
Warning
|
Since Each |
Examples
Resolve logEvent.getThrown().getClass().getCanonicalName()
:
{
"$resolver": "exception",
"field": "className"
}
Resolve the stack trace into a list of StackTraceElement
objects:
{
"$resolver": "exception",
"field": "stackTrace"
}
Resolve the stack trace into a string field:
{
"$resolver": "exception",
"field": "stackTrace",
"stackTrace": {
"stringified": true
}
}
Resolve the stack trace into a string field such that the content will be truncated after the given point matcher:
{
"$resolver": "exception",
"field": "stackTrace",
"stackTrace": {
"stringified": {
"truncation": {
"suffix": "... [truncated]",
"pointMatcherStrings": ["at javax.servlet.http.HttpServlet.service"]
}
}
}
}
Resolve the stack trace into an object described by the provided stack trace element template:
{
"$resolver": "exception",
"field": "stackTrace",
"stackTrace": {
"elementTemplate": {
"class": {
"$resolver": "stackTraceElement",
"field": "className"
},
"method": {
"$resolver": "stackTraceElement",
"field": "methodName"
},
"file": {
"$resolver": "stackTraceElement",
"field": "fileName"
},
"line": {
"$resolver": "stackTraceElement",
"field": "lineNumber"
}
}
}
}
See Stack Trace Element Templates for further details on resolvers available
for StackTraceElement
templates.
exceptionRootCause
Resolves the fields of the innermost Throwable
returned by
logEvent.getThrown()
. Its syntax and garbage-footprint are identical to the
exception
resolver.
level
config = field , [ severity ]
field = "field" -> ( "name" | "severity" )
severity = severity-field
severity-field = "field" -> ( "keyword" | "code" )
Resolves the fields of the logEvent.getLevel()
.
Examples
Resolve the level name:
{
"$resolver": "level",
"field": "name"
}
Resolve the Syslog severity keyword:
{
"$resolver": "level",
"field": "severity",
"severity": {
"field": "keyword"
}
}
Resolve the Syslog severity code:
{
"$resolver": "level",
"field": "severity",
"severity": {
"field": "code"
}
}
logger
config = "field" -> ( "name" | "fqcn" )
Resolves logEvent.getLoggerFqcn()
and logEvent.getLoggerName()
.
Examples
Resolve the logger name:
{
"$resolver": "logger",
"field": "name"
}
Resolve the logger’s fully qualified class name:
{
"$resolver": "logger",
"field": "fqcn"
}
main
config = ( index | key )
index = "index" -> number
key = "key" -> string
Performs Main Argument Lookup for the
given index
or key
.
Examples
Resolve the 1st main()
method argument:
{
"$resolver": "main",
"index": 0
}
Resolve the argument coming right after --userId
:
{
"$resolver": "main",
"key": "--userId"
}
map
Resolves MapMessage
s. See Map Resolver Template
for details.
marker
config = "field" -> ( "name" | "parents" )
Resolves logEvent.getMarker()
.
Examples
Resolve the marker name:
{
"$resolver": "marker",
"field": "name"
}
Resolve the names of the marker’s parents:
{
"$resolver": "marker",
"field": "parents"
}
mdc
Resolves Mapped Diagnostic Context (MDC), aka. Thread Context Data. See Map Resolver Template for details.
Warning
|
|
message
config = [ stringified ] , [ fallbackKey ]
stringified = "stringified" -> boolean
fallbackKey = "fallbackKey" -> string
Resolves logEvent.getMessage()
.
Warning
|
For simple string messages, the resolution is performed without allocations.
For |
Examples
Resolve the message into a string:
{
"$resolver": "message",
"stringified": true
}
Resolve the message such that if it is an ObjectMessage
or a
MultiformatMessage
with JSON support, its type (string, list, object, etc.)
will be retained:
{
"$resolver": "message"
}
Given the above configuration, a SimpleMessage
will generate a "sample log
message"
, whereas a MapMessage
will generate a {"action": "login",
"sessionId": "87asd97a"}
. Certain indexed log storage systems (e.g.,
Elasticsearch) will not allow both values
to coexist due to type mismatch: one is a string
while the other is an object
.
Here one can use a fallbackKey
to work around the problem:
{
"$resolver": "message",
"fallbackKey": "formattedMessage"
}
Using this configuration, a SimpleMessage
will generate a
{"formattedMessage": "sample log message"}
and a MapMessage
will generate a
{"action": "login", "sessionId": "87asd97a"}
. Note that both emitted JSONs are
of type object
and have no type-conflicting fields.
messageParameter
config = [ stringified ] , [ index ]
stringified = "stringified" -> boolean
index = "index" -> number
Resolves logEvent.getMessage().getParameters()
.
Warning
|
Regarding garbage footprint, |
Examples
Resolve the message parameters into an array:
{
"$resolver": "messageParameter"
}
Resolve the string representation of all message parameters into an array:
{
"$resolver": "messageParameter",
"stringified": true
}
Resolve the first message parameter:
{
"$resolver": "messageParameter",
"index": 0
}
Resolve the string representation of the first message parameter:
{
"$resolver": "messageParameter",
"index": 0,
"stringified": true
}
ndc
config = [ pattern ]
pattern = "pattern" -> string
Resolves the Nested Diagnostic Context (NDC), aka. Thread Context Stack,
String[]
returned by logEvent.getContextStack()
.
Examples
Resolve all NDC values into a list:
{
"$resolver": "ndc"
}
Resolve all NDC values matching with the pattern
regex:
{
"$resolver": "ndc",
"pattern": "user(Role|Rank):\\w+"
}
pattern
config = pattern , [ stackTraceEnabled ]
pattern = "pattern" -> string
stackTraceEnabled = "stackTraceEnabled" -> boolean
Resolver delegating to PatternLayout
.
The default value of stackTraceEnabled
is inherited from the parent
JsonTemplateLayout
.
Examples
Resolve the string produced by %p %c{1.} [%t] %X{userId} %X %m%ex
pattern:
{
"$resolver": "pattern",
"pattern": "%p %c{1.} [%t] %X{userId} %X %m%ex"
}
source
config = "field" -> (
"className" |
"fileName" |
"methodName" |
"lineNumber" )
Resolves the fields of the StackTraceElement
returned by
logEvent.getSource()
.
Note that this resolver is toggled by
log4j.layout.jsonTemplate.locationInfoEnabled
property.
Examples
Resolve the line number:
{
"$resolver": "source",
"field": "lineNumber"
}
thread
config = "field" -> ( "name" | "id" | "priority" )
Resolves logEvent.getThreadId()
, logEvent.getThreadName()
,
logEvent.getThreadPriority()
.
Examples
Resolve the thread name:
{
"$resolver": "thread",
"field": "name"
}
timestamp
config = [ patternConfig | epochConfig ]
patternConfig = "pattern" -> ( [ format ] , [ timeZone ] , [ locale ] )
format = "format" -> string
timeZone = "timeZone" -> string
locale = "locale" -> (
language |
( language , "_" , country ) |
( language , "_" , country , "_" , variant )
)
epochConfig = "epoch" -> ( unit , [ rounded ] )
unit = "unit" -> (
"nanos" |
"millis" |
"secs" |
"millis.nanos" |
"secs.nanos" |
)
rounded = "rounded" -> boolean
Resolves logEvent.getInstant()
in various forms.
Examples
Configuration |
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
|
Map Resolver Template
ReadOnlyStringMap
is Log4j’s Map<String, Object>
equivalent with
garbage-free accessors and heavily employed throughout the code base. It is the
data structure backing both Mapped Diagnostic Context (MDC), aka. Thread Context
Data and MapMessage
implementations. Hence template resolvers for both of
these are provided by a single backend: ReadOnlyStringMapResolver
. Put another
way, both mdc
and map
resolvers support identical configuration, behaviour,
and garbage footprint, which are detailed below.
config = singleAccess | multiAccess
singleAccess = key , [ stringified ]
key = "key" -> string
stringified = "stringified" -> boolean
multiAccess = [ pattern ] , [ replacement ] , [ flatten ] , [ stringified ]
pattern = "pattern" -> string
replacement = "replacement" -> string
flatten = "flatten" -> ( boolean | flattenConfig )
flattenConfig = [ flattenPrefix ]
flattenPrefix = "prefix" -> string
singleAccess
resolves a single field, whilst multiAccess
resolves a
multitude of fields. If flatten
is provided, multiAccess
merges the fields
with the parent, otherwise creates a new JSON object containing the values.
Enabling stringified
flag converts each value to its string representation.
Regex provided in the pattern
is used to match against the keys. If provided,
replacement
will be used to replace the matched keys. These two are
effectively equivalent to Pattern.compile(pattern).matcher(key).matches()
and
Pattern.compile(pattern).matcher(key).replaceAll(replacement)
calls.
Warning
|
Regarding garbage footprint,
Writing certain non-primitive values (e.g., |
"$resolver"
is left out in the following examples, since it is to be
defined by the actual resolver, e.g., map
, mdc
.
Resolve the value of the field keyed with user:role
:
{
"$resolver": "…",
"key": "user:role"
}
Resolve the string representation of the user:rank
field value:
{
"$resolver": "…",
"key": "user:rank",
"stringified": true
}
Resolve all fields into an object:
{
"$resolver": "…"
}
Resolve all fields into an object such that values are converted to string:
{
"$resolver": "…",
"stringified": true
}
Resolve all fields whose keys match with the user:(role|rank)
regex into an
object:
{
"$resolver": "…",
"pattern": "user:(role|rank)"
}
Resolve all fields whose keys match with the user:(role|rank)
regex into an
object after removing the user:
prefix in the key:
{
"$resolver": "…",
"pattern": "user:(role|rank)",
"replacement": "$1"
}
Merge all fields whose keys are matching with the user:(role|rank)
regex into
the parent:
{
"$resolver": "…",
"flatten": true,
"pattern": "user:(role|rank)"
}
After converting the corresponding field values to string, merge all fields to
parent such that keys are prefixed with _
:
{
"$resolver": "…",
"stringified": true,
"flatten": {
"prefix": "_"
}
}
Stack Trace Element Templates
exception
and
exceptionRootCause
event template resolvers can
serialize an exception stack trace (i.e., StackTraceElement[]
returned by
Throwable#getStackTrace()
) into a JSON array. While doing so, JSON templating
infrastructure is used again.
stackTraceElement[Uri]
describes the JSON structure JsonTemplateLayout
uses
to format StackTraceElement
s. The default configuration (accessible by
log4j.layout.jsonTemplate.stackTraceElementTemplate[Uri]
property) is set to
classpath:StackTraceElementLayout.json
provided by the
log4j-layout-template-json
artifact:
{
"class": {
"$resolver": "stackTraceElement",
"field": "className"
},
"method": {
"$resolver": "stackTraceElement",
"field": "methodName"
},
"file": {
"$resolver": "stackTraceElement",
"field": "fileName"
},
"line": {
"$resolver": "stackTraceElement",
"field": "lineNumber"
}
}
The allowed template configuration syntax is as follows:
config = "field" -> (
"className" |
"fileName" |
"methodName" |
"lineNumber" )
All above accesses to StackTraceElement
is garbage-free.
Extending
JsonTemplateLayout
relies on Log4j plugin system to build
up the features it provides. This enables feature customization a breeze for
users. As of this moment, following features are implemented by means of
plugins:
-
Event template resolvers (e.g.,
exception
,message
,level
event template resolvers) -
Event template interceptors (e.g., injection of
eventTemplateAdditionalField
) -
Recycler factories
Following sections cover these in detail.
Plugin Preliminaries
Log4j plugin system is the de facto extension mechanism embraced by various
Log4j components, including JsonTemplateLayout
. Plugins make it possible
for extensible components receive feature implementations without any explicit
links in between. It is analogous to a
dependency injection
framework, but curated for Log4j-specific needs.
In a nutshell, you annotate your classes with @Plugin
and their (static
)
creator methods with @PluginFactory
. Last, you inform the Log4j plugin system
to discover these custom classes. This can be done either using packages
declared in your Log4j configuration or by various other ways described in
plugin system documentation.
Extending Event Resolvers
All available event template resolvers are simple
plugins employed by JsonTemplateLayout
. To add new ones, one just needs to
create their own EventResolver
and instruct its injection via a
@Plugin
-annotated EventResolverFactory
class.
For demonstration purposes, below we will create a randomNumber
event resolver.
Let’s start with the actual resolver:
package com.acme.logging.log4j.layout.template.json;
import org.apache.logging.log4j.core.LogEvent;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolver;
import org.apache.logging.log4j.layout.template.json.util.JsonWriter;
/**
* Resolves a random floating point number.
*
* <h3>Configuration</h3>
*
* <pre>
* config = ( [ range ] )
* range = number[]
* </pre>
*
* {@code range} is a number array with two elements, where the first number
* denotes the start (inclusive) and the second denotes the end (exclusive).
* {@code range} is optional and by default set to {@code [0, 1]}.
*
* <h3>Examples</h3>
*
* Resolve a random number between 0 and 1:
*
* <pre>
* {
* "$resolver": "randomNumber"
* }
* </pre>
*
* Resolve a random number between -0.123 and 0.123:
*
* <pre>
* {
* "$resolver": "randomNumber",
* "range": [-0.123, 0.123]
* }
* </pre>
*/
public final class RandomNumberResolver implements EventResolver {
private final double loIncLimit;
private final double hiExcLimit;
RandomNumberResolver(final TemplateResolverConfig config) {
final List<Number> rangeArray = config.getList("range", Number.class);
if (rangeArray == null) {
this.loIncLimit = 0D;
this.hiExcLimit = 1D;
} else if (rangeArray.size() != 2) {
throw new IllegalArgumentException(
"range array must be of size two: " + config);
} else {
this.loIncLimit = rangeArray.get(0).doubleValue();
this.hiExcLimit = rangeArray.get(1).doubleValue();
if (loIncLimit > hiExcLimit) {
throw new IllegalArgumentException("invalid range: " + config);
}
}
}
static String getName() {
return "randomNumber";
}
@Override
public void resolve(
final LogEvent value,
final JsonWriter jsonWriter) {
final double randomNumber =
loIncLimit + (hiExcLimit - loIncLimit) * Math.random();
jsonWriter.writeNumber(randomNumber);
}
}
Next create a EventResolverFactory
class to register RandomNumberResolver
into the Log4j plugin system.
RandomNumberResolver
into the Log4j plugin systempackage com.acme.logging.log4j.layout.template.json;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.config.plugins.PluginFactory;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverContext;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverFactory;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolver;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverConfig;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverFactory;
/**
* {@link RandomNumberResolver} factory.
*/
@Plugin(name = "RandomNumberResolverFactory", category = TemplateResolverFactory.CATEGORY)
public final class RandomNumberResolverFactory implements EventResolverFactory {
private static final RandomNumberResolverFactory INSTANCE =
new RandomNumberResolverFactory();
private RandomNumberResolverFactory() {}
@PluginFactory
public static RandomNumberResolverFactory getInstance() {
return INSTANCE;
}
@Override
public String getName() {
return RandomNumberResolver.getName();
}
@Override
public RandomNumberResolver create(
final EventResolverContext context,
final TemplateResolverConfig config) {
return new RandomNumberResolver(config);
}
}
Almost complete. Last, we need to inform the Log4j plugin system to discover these custom classes:
randomNumber
resolver<?xml version="1.0" encoding="UTF-8"?>
<Configuration packages="com.acme.logging.log4j.layout.template.json">
<!-- ... -->
<JsonTemplateLayout>
<EventTemplateAdditionalField
key="id"
format="JSON"
value='{"$resolver": "randomNumber", "range": [0, 1000000]}'/>
</JsonTemplateLayout>
<!-- ... -->
</Configuration>
All available event template resolvers are located in
org.apache.logging.log4j.layout.template.json.resolver
package. It is a fairly
rich resource for inspiration while implementing new resolvers.
Intercepting the Template Resolver Compiler
JsonTemplateLayout
allows interception of the template resolver compilation,
which is the process converting a template into a Java function performing the
JSON serialization. This interception mechanism is internally used to implement
eventTemplateRootObjectKey
and eventTemplateAdditionalField
features. In a
nutshell, one needs to create a @Plugin
-annotated class extending from
EventResolverInterceptor
interface.
To see the interception in action, check out the EventRootObjectKeyInterceptor
class which is responsible for implementing the eventTemplateRootObjectKey
feature:
eventTemplateRootObjectKey
, if presentimport org.apache.logging.log4j.layout.template.json.resolver.EventResolverContext;
import org.apache.logging.log4j.layout.template.json.resolver.EventResolverInterceptor;
import org.apache.logging.log4j.layout.template.json.resolver.TemplateResolverInterceptor;
/**
* Interceptor to add a root object key to the event template.
*/
@Plugin(name = "EventRootObjectKeyInterceptor", category = TemplateResolverInterceptor.CATEGORY)
public class EventRootObjectKeyInterceptor implements EventResolverInterceptor {
private static final EventRootObjectKeyInterceptor INSTANCE =
new EventRootObjectKeyInterceptor();
private EventRootObjectKeyInterceptor() {}
@PluginFactory
public static EventRootObjectKeyInterceptor getInstance() {
return INSTANCE;
}
@Override
public Object processTemplateBeforeResolverInjection(
final EventResolverContext context,
final Object node) {
String eventTemplateRootObjectKey = context.getEventTemplateRootObjectKey();
return eventTemplateRootObjectKey != null
? Collections.singletonMap(eventTemplateRootObjectKey, node)
: node;
}
}
Here, processTemplateBeforeResolverInjection()
method checks if the user has
provided an eventTemplateRootObjectKey
. If so, it wraps the root node
with a
new object; otherwise, returns the node
as is. Note that node
refers to the
root Java object of the event template read by JsonReader
.
Extending Recycler Factories
recyclerFactory
input String
read from the layout configuration is converted
to a RecyclerFactory
using the default RecyclerFactoryConverter
extending
from TypeConverter<RecyclerFactory>
. If one wants to change this behavior,
they simply need to add their own TypeConverter<RecyclerFactory>
implementing
Comparable<TypeConverter<?>>
to prioritize their custom converter.
TypeConverter
for RecyclerFactory
package com.acme.logging.log4j.layout.template.json;
import org.apache.logging.log4j.core.config.plugins.Plugin;
import org.apache.logging.log4j.core.config.plugins.convert.TypeConverter;
import org.apache.logging.log4j.core.config.plugins.convert.TypeConverters;
@Plugin(name = "AcmeRecyclerFactoryConverter", category = TypeConverters.CATEGORY)
public final class AcmeRecyclerFactoryConverter
implements TypeConverter<RecyclerFactory>, Comparable<TypeConverter<?>> {
@Override
public RecyclerFactory convert(final String recyclerFactorySpec) {
return AcmeRecyclerFactory.ofSpec(recyclerFactorySpec);
}
@Override
public int compareTo(final TypeConverter<?> ignored) {
return -1;
}
}
Here note that compareTo()
always returns -1 to rank it higher compared to
other matching converters.
Features
Below is a feature comparison matrix between JsonTemplateLayout
and
alternatives.
Feature |
|
|||
Java version |
8 |
8 |
8 |
6 |
Dependencies |
None |
Jackson |
None |
None |
Schema customization? |
✓ |
✕ |
✕ |
✕ |
Timestamp customization? |
✓ |
✕ |
✕ |
✕ |
(Almost) garbage-free? |
✓ |
✕ |
✓ |
✓ |
Custom typed |
✓ |
✕ |
✕ |
?[1] |
Custom typed |
✓ |
✕ |
✕ |
✕ |
Rendering stack traces as array? |
✓ |
✓ |
✕ |
✓ |
Stack trace truncation? |
✓ |
✕ |
✕ |
✕ |
JSON pretty print? |
✕ |
✓ |
✕ |
✕ |
Additional string fields? |
✓ |
✓ |
✓ |
✓ |
Additional JSON fields? |
✓ |
✕ |
✕ |
✕ |
Custom resolvers? |
✓ |
✕ |
✕ |
✕ |
F.A.Q.
Are lookups supported in templates?
Yes, lookups (e.g., ${java:version}
,
${env:USER}
, ${date:MM-dd-yyyy}
) are supported in string
literals of templates. Though note that they are not garbage-free.
Are recursive collections supported?
No. Consider a Message
containing a recursive value as follows:
Object[] recursiveCollection = new Object[1];
recursiveCollection[0] = recursiveCollection;
While the exact exception might vary, you will most like get a
StackOverflowError
for trying to render recursiveCollection
into a
String
. Note that this is also the default behaviour for other Java standard
library methods, e.g., Arrays.toString()
. Hence mind self references while
logging.
Is JsonTemplateLayout
garbage-free?
Yes, if the garbage-free layout behaviour toggling properties
log4j2.enableDirectEncoders
and log4j2.garbagefreeThreadContextMap
are
enabled. Take into account the following caveats:
-
The configured recycling strategy might not be garbage-free.
-
Since
Throwable#getStackTrace()
clones the originalStackTraceElement[]
, access to (and hence rendering of) stack traces are not garbage-free. -
Serialization of
MapMessage
s andObjectMessage
s are mostly garbage-free except for certain types (e.g.,BigDecimal
,BigInteger
,Collection
s, exceptList
). -
Lookups (that is,
${…}
variables) are not garbage-free.
Don’t forget to check out the notes on garbage footprint of resolvers you employ in templates.
ObjectMessage
s and if Jackson is in the classpath.