Telling the story right: efficient logging in Go

What to log #

The first thing to figure out is what to log, which falls into roughly three categories:

First there are errors. These should always be logged at the ERROR level, unless you have a good reason not to.

Be sure to include the stack trace (using the approach from the previous post) and the relevant conditions under which the error occurred. This might get logged like this:

{"@t": "2026-03-12T03:44:57.8532799Z", "@mt": "Failed to get user: connection refused", "@x": "stacktrace..", "@l": "error"}

Next are business events and important state transitions. These are generally logged at the INFO level. Again, include relevant context, most importantly who performed the action (for example, by including the user ID):

{"@t": "2026-03-12T03:44:57.8532799Z", "@mt": "Processing order", "orderID": 7453025715642961920, "userID": 2047126549831876608, "@l": "informational"}

The last category is communication with other systems: your database, third-party services, and client applications. Levels here vary by importance. Inbound HTTP requests and outbound third-party calls are usually worth INFO, other communication may be too noisy for anything but DEBUG:

{"@t": "2026-03-12T03:44:57.8532799Z", "@mt": "POST /api/orders returned 201 in 142ms", "method": "POST", "path": "/api/orders", "status": 201, "duration_ms": 142, "@l": "debug"}

Keeping these three categories in mind, there are a few things I see teams getting wrong when deciding what to log:

• Logging sensitive details. Examples include application secrets, user credentials, and PII.
• Running the production environment at the wrong log level. INFO is generally the right default.
• Prematurely optimising log volume. There’s a balance to be struck here, but logging too little hurts more for the vast majority of teams.

Logging formats #

When you’ve figured out what to log, the next step is to decide your log format(s). We’ll consider two formats: structured logs and plain text logs.

Structured logs, like the examples in the previous section, are essential in production. As your application grows, you’ll need to filter and query your logs to reason about them — something that’s vastly easier with structured data.

Locally, you want plain text logs like the ones shown in the screenshot below. They’re harder to query but easier to read compared to to structured logs. Colour and formatting lets you scan a running stream while you work, and a bit of pizzazz makes the dev loop more pleasant.

One thing most people miss on this topic is to keep plain text logs running in production, alongside the structured ones. When your log ingestion pipeline breaks (and it will), the plain text logs are what you’ll have to fall back on and may even help you fix the broken pipeline.

Implementation using slog #

Below I’ll walk through how to implement a production-ready logging setup with slog following best practices. The patterns here apply regardless of your logging library; slog is just the lowest-friction choice because it’s in the standard library. Alternatives like zap work the same way, with broadly similar APIs and reportedly better performance.

I’ll assume some familiarity with slog’s basics (but the standard library docs are a good primer if you need one).

slog provides two handler types out of the box: JSONHandler for structured logs and TextHandler for plain text logs. They’re minimal: they won’t pick up the stack traces attached to our errors, and the text handler doesn’t provide any highlighting. We’ll be making our own versions that do provide these features.

Structured logs using the SeqHandler #

Our custom structured log handler, SeqHandler, will take a slog.Record and send it to Seq, a self-hostable log aggregation and search platform.

The full handler is available on gist but the most interesting parts can be found below. It shows how it converts a slog.Record into a CLEF event, and how it handles errors by extracting the stack trace into the @x field that Seq displays as exception data, and (where possible) attaching the response body of HTTP errors. Reminder: be careful with the latter from a security perspective.

func (handler *SeqHandler) recordToSeqEvent(record slog.Record) (map[string]any, error) {
	seqLevel, err := slogLevelToSeqLevel(record.Level)
	if err != nil {
		return map[string]any{}, err
	}

	event := map[string]any{
		"@t": record.Time.Format(time.RFC3339Nano),
		"@l": seqLevel,
		"@m": record.Message,
	}

	eventAttrs := map[string]any{}

	record.Attrs(func(attr slog.Attr) bool {
		addAttr(eventAttrs, attr)
		return true
	})

	// ..

	return event, nil
}

func addAttr(event map[string]any, attr slog.Attr) {
	if attr.Key == "err" {
		if err, ok := attr.Value.Any().(error); ok {
			addErrorAttrs(event, err)
			return
		}
	}
	event[attr.Key] = attr.Value.Any()
}

func addErrorAttrs(event map[string]any, err error) {
	event["@x"] = fmt.Sprintf("%+v", err)
	httpError, ok := errors.Cause(err).(HTTPError)
	if ok {
		if responseBody, err := httpError.ReadBody(); err == nil {
			event["respBody"] = responseBody
		}
	}
}

Possible improvements to this handler can be adding support for traces and spans¹, and batching records before sending them to Seq to reduce network overhead.

Human readable logs using the text handler #

Our custom text handler, TextHandler, wraps any given slog.Handler. I like to pair it with charmbracelet/log, which logs to your terminal in a colourful, human-readable format by default (including support for attributes) and allows for extensive customisation:

func newConsoleHandler(level Level) slog.Handler {
    charmlogger := charmlog.NewWithOptions(os.Stdout, charmlog.Options{
        ReportTimestamp: true,
        Level:           level.Charm,
    })
    return NewTextHandler(charmlogger)
}

The full text handler is available on gist. It does two main things, recursively, for each slog.Attr:

1. Replaces tabs in attribute strings with four spaces, for cleaner formatting (especially when used together with charmbracelet/log).
2. Adds the stack trace of any error instances, including the response body of HTTP errors where possible.

As with the SeqHandler, this can be further improved by adding support for tracing and spans¹.

Fallback to plain text in production #

Earlier in the post, I argued for keeping plain text logs running in production alongside structured ones by not only running both handlers in parallel, but also falling back to text when the structured pipeline breaks, and logging the breakage itself. A small wrapper around the SeqHandler does the job:

type seqHandlerWithFallback struct {
	*seqHandler
	fallbackHandler slog.Handler
}

func (handler *seqHandlerWithFallback) Handle(ctx context.Context, record slog.Record) error {
	err := handler.seqHandler.Handle(ctx, record)
	if err != nil {
		seqErrRecord := slog.NewRecord(record.Time, slog.LevelError, "Failed to post event to Seq", 1)
		seqErrRecord.Add("err", err)
		_ = handler.fallbackHandler.Handle(ctx, seqErrRecord)
		fallbackErr := handler.fallbackHandler.Handle(ctx, record)
		if fallbackErr != nil {
			return errors.Join(err, fallbackErr)
		}
	}
	return nil
}

func (handler *seqHandlerWithFallback) WithAttrs(attrs []slog.Attr) slog.Handler {
	return &seqHandlerWithFallback{
		seqHandler:      handler.seqHandler.WithAttrs(attrs),
		fallbackHandler: handler.fallbackHandler.WithAttrs(attrs),
	}
}

func (handler *seqHandlerWithFallback) WithGroup(name string) slog.Handler {
	return &seqHandlerWithFallback{
		seqHandler:      handler.seqHandler.WithGroup(name),
		fallbackHandler: handler.fallbackHandler.WithGroup(name),
	}
}

On failure, the wrapper writes two records to the fallback handler: an error record explaining that Seq is unreachable, and the original record that would otherwise have been lost if you’re running no other handlers.

Wiring it together #

As discussed previously, running multiple log handlers in parallel can be beneficial to provide redundancy. slog provides the slog.NewMultiHandler function, which takes one or more handlers, for exactly this purpose. Log formats can be specified together with their levels in your application’s configuration. You can then proceed to loop through this mapping and create a new handler for each entry:

 // New creates a new slog.Logger with the given level and format.
func New(formatsWithLevels map[Format]Level) (*slog.Logger, error) {
	var handlers []slog.Handler
	for format, level := range formatsWithLevels {
		handler, err := newHandler(format, level)
		if err != nil {
			return nil, err
		}
		handlers = append(handlers, handler)
	}
	multiHandler := slog.NewMultiHandler(handlers...)
	return slog.New(multiHandler), nil
}

A proper Enabled implementation on each handler ensures it only fires for its configured level.

Wrapping up #

Errors in Go are useless without their context. Your logging implementation exists to surface that context. Get both right and your application becomes much easier to maintain.

This is part 2 of a series. Part 1 covers Go’s error model and why most teams get it wrong.