🎄Structured logging and why you need it in 2024🌟

Back to 2023 Advent Calendar

What is structured logging?

Structured logging is a way to log your data in a structured way. Instead of logging into plain text files, you log into json files (or JSON formatted strings sent to log aggregator).

e.g instead of logging this:

2023-12-10 12:00:00,000 INFO  [main] com.example.MyClass - User 123 logged in

You log this:

{
    "timestamp": "2023-12-10 12:00:00,000",
    "level": "INFO",
    "thread": "main",
    "logger": "com.example.MyClass",
    "message": "User logged in",
    "user_id": 123,
}

What can you do with structured logging?

With structured logging you can do a lot of things. Here are a few examples:

1. You can search your logs in a log aggregator. e.g Kibana or Azure Log Analytics.
2. You can build dashboards and alerts based on your logs.
3. You can even alert based on specific conditions.

How to do structured logging?

There are many ways to do structured logging. Here are a few examples:

1. Log as JSON either via a library or just serialize your message as JSON (e.g using Jackson).
2. Use collectors like logstash or fluentd to collect your logs and send them to a log aggregator (e.g Elasticsearch or Kusto).
3. Use a query language / dashboard to search your logs and build dashboards (e.g Kibana or Azure Log Analytics).