Custom Formating

Table of contents

Introduction

Probably the single biggest omission from the version of Flogger released by Google is a lack of configurable message format. Flogger Next fixes this by providing logger backend implementations which can take advantage of an easily configured customizable message formatter.

To change the message format for Flogger backends, simply provide a Flogger Next option of the form:

flogger.message_formatter.pattern=<message template>

This will define the message format for Flogger, but will not affect any additional formatting added by the underling logging system. As such, it might also be beneficial to modify the underlying formatting to avoid duplicate message elements etc.

Message Pattern Syntax

The message syntax is based on simple token substitution, where known tokens (e.g. %{message}) are replaced by the specific element value, and other text is treated literally.

A simple example to emit the log message followed by a space and any metadata, might be:

flogger.message_formatter.pattern=${message} %{metadata}

However, for tokens with optional values (e.g. metadata keys which may not be set for a specific log event) there is the ability to set a prefix and suffix for a token, which will only be emitted if the token is substituted. This makes it easy to add a bit of structure to log messages while avoiding additional separators or empty lists (e.g. []).

This is achieved by adding optional prefix and suffix text in the token descriptor, such as %{token-name/prefix-text} or %{token-name/prefix-text/suffix-text}

For example, to avoid having a trailing space in the above example, it is better to write:

flogger.message_formatter.pattern=${message}%{metadata/ }

which only adds a space after the message (as a prefix to the metadata) if metadata is present.

Extending this to wrap metadata with array-like formatting with [, ], you just need:

flogger.message_formatter.pattern=${message}%{metadata/ [/]}

which adds surrounding [, ] to any metadata present for that log event.

Prefix and suffix text is arbitrary, and you can use \ to escape meta-characters such as }, / and \ itself.

Known Tokens

The formatting of token values is handled by a series of plugin classes. This permits common cases to be handled by configuring the default plugin, but allows most plugins to be replaced, if needed, by user code.

Tokens With Non-Optional Values

The following tokens always emit a value, so prefix or suffix text is not allowed.

  • %{message}:
    • The formatted base message passed to the log() method. This is formatted according to the chosen backend implementation, and cannot be modified directly by the pattern formatter.
    • This is a built-in format, and cannot be replaced with a plugin.
  • %{timestamp}:
    • The timestamp of the log event. By default, this is formatted according to ISO-8601 format, but this can be modified via the flogger.message_formatter.timestamp.pattern option.
    • See DefaultTimestampFormatter for more details.
    • This plugin can be changed via flogger.message_formatter.timestamp.impl
  • %{level}:
    • The level of log event. By default, this is formatted using the inbuilt name (e.g. “INFO”) but can be set to use the internationalized name via the flogger.message_formatter.level.use_localized_name option.
    • See DefaultLevelFormatter for more details.
    • This plugin can be changed via flogger.message_formatter.level.impl
  • %{location}:
    • This emits the log site information for the log event in the form <class-name>#<method-name>. Currently, there are no options for the default level formatter.
    • See DefaultLocationFormatter for more details.
    • This plugin can be changed via flogger.message_formatter.location.impl

Tokens With Optional Values

Tokens with optional values can have prefix and suffix text provided.

  • %{metadata}
    • This emits a sequence of space-separated metadata key/value pairs in the form <key-label>=<value>.
    • Individual keys can be excluded from appearing in the output by listing MetadataKey fields in the flogger.message_formatter.metadata.ignore options list (see below).
    • The behaviour of individual keys is customizable via MetadataKey subclasses, and not this plugin.
    • This is a built-in token and cannot be replaced with a plugin.
  • %{key.<key-name>}
    • This emits the value of the MetadataKey (if present) that’s linked from the flogger.message_formatter.metadata.key.<key-name> option, without the key label or any other formatting.
    • This is a built-in token and cannot be replaced with a plugin.

Metadata Keys

For the %{key.xxx} tokens, and ignored keys in the %{metadata} token, you need to specify a MetadataKey by name. To do this, specify the field of the metadata key in the form <class-name>#<field-name>. However, there are a couple of important notes about metadata keys to be aware of.

  1. To reference a key by name, the key must be a public static final field of a public class.
  2. To avoid triggering unwanted class initialization or reentrant logging early in an application’s lifetime, metadata keys should be held in a class of their own (e.g. a nested Key class).

The following is an example of a metadata key field suitable for loading by Flogger Next:

package org.something.myapp;

import com.google.common.flogger.MetadataKey;

final class MyApplicationClass {
  /**
   * Public metadata keys for MyApplication.
   *
   * <p>Nested class to avoid static initialization when the key is loaded by the formatter.
   */
  public static final class Key {
    /** Shows the current task ID. */
    public static final MetadataKey<String> TASK_ID = MetadataKey.single("task", String.class);
  }
}

With the above setup, the TASK_ID can be referenced via:

org.something.myapp.MyApplicationClass$Key#TASK_ID

Custom Metadata Labels

If a custom MetadataKey subclass emits multiple values with non-default labels, the value can be referenced by appending the custom label name to the end of the key identifier, separated by a :.

For example, a custom GPS metadata key (with the base label gps) might emit latitude and longitude separately as gps.lat and gps.lng. In this case the values can be identified for custom formatting via:

org.something.myapp.MyApplicationClass$Key#GPS_COORD:gps.lat
org.something.myapp.MyApplicationClass$Key#GPS_COORD:gps.lng

Formatter Plugins

Almost all Flogger Next formatting, including the overall pattern formatter, is controlled via Flogger Next’s plugin mechanism. This lets implementation be replaced by specifying a class name to an impl option.

A formatter plugin class must adhere to two basic API contraints:

  1. Implement the LogMessageFormatter interface for appending to the log format buffer.
    • Since this code is always called for every log statement, the plugin code MUST NOT do any logging of its own or call any code which might do logging.
  2. Provide a public constructor which accepts an Options instance to configure the plugin.
    • Options passed to a plugin will be scoped to the base name of the plugin (i.e. the options for flogger.message_formatter.xxxx.impl will be scoped to flogger.message_formatter.xxxx).

Formatter plugins can be specified by one of the following Flogger Next options:

  • flogger.message_formatter.impl: The overall message formatter class. If this is replaced, none of the other formatter plugins will have any effect by default.
  • flogger.message_formatter.level.impl: The log level formatter class.
  • flogger.message_formatter.location.impl: The log site location formatter class.
  • flogger.message_formatter.timestamp.impl: The log timestamp formatter class.

Installation

JDK logging backend (replaces the com.google.flogger:flogger-system-backend dependency):

<dependency>
  <groupId>net.goui.flogger.next</groupId>
  <artifactId>backend-system</artifactId>
  <version>${flogger-next.version}</version>
</dependency>

Log4J 2 backend (replaces the com.google.flogger:flogger-log4j2-backend dependency):

<dependency>
  <groupId>net.goui.flogger.next</groupId>
  <artifactId>backend-log4j</artifactId>
  <version>${flogger-next.version}</version>
</dependency>