Logging Framework

Design a small logging library like Log4j, with levels, several output targets and pluggable formats. Logging must never slow down or crash the app.

Design a logging library for our services. Code calls `logger.info(message)` or `logger.error(message, exception)`. Messages below a configured level are ignored. Each message can go to the console, a file or both, in plain text or JSON.

Questions to ask first

  • Which levels? Assume DEBUG, INFO, WARN and ERROR, in that order.
  • Where can logs go? Console and file now. A database or a remote service later, so new targets must be easy to add.
  • Can each target have its own level and format? Yes. For example, the console shows DEBUG in plain text and the file keeps WARN and above in JSON.
  • May logging slow the request? No. Writing to disk must not block business threads for long.
  • Do we need one logger per class? Yes. Named loggers like `OrderService` come from a factory and are reused.

Requirements

  • Log a message at a level, with the time, logger name, thread name and an optional exception.
  • Drop messages below the logger's minimum level, cheaply.
  • Send each message to every attached appender (an output target), each with its own level and format.
  • Add new appenders and formats without changing Logger.
  • Return the same Logger for the same name.
  • Never throw from a log call, even when an appender fails.

Out of scope

  • Rotating log files by size or date (an extension).
  • Shipping logs to a central server.
  • Reading configuration from XML or YAML files.

Entities

  • Logger: name, minLevel, appenders; log(level, msg, t), info(msg), error(msg, t)
  • LoggerFactory: getLogger(name), keeps one Logger per name
  • LogLevel: DEBUG, INFO, WARN, ERROR
  • LogRecord: time, level, logger, thread, message, error
  • Appender: append(record); ConsoleAppender, FileAppender
  • AbstractAppender: level check and formatting once; write(line) left to subclasses
  • Formatter: format(record); PatternFormatter, JsonFormatter

Classes in the model design

  • LoggerFactory (class)
  • Logger (class)
  • LogLevel (enum)
  • LogRecord (class)
  • Appender (interface)
  • AbstractAppender (abstract)
  • ConsoleAppender (class)
  • FileAppender (class)
  • Formatter (interface)
  • PatternFormatter (class)
  • JsonFormatter (class)

The main use case, step by step

  1. `OrderService` gets its logger once with `factory.getLogger("OrderService")`. The factory uses `computeIfAbsent`, so every call with that name returns the same Logger.
  2. The code calls `log.info("Order 991 placed")`. Logger compares INFO with its minimum level first. If the level is too low, it returns at once, before building any object.
  3. Logger builds one immutable `LogRecord` with the time, level, name, thread and message. Immutable means it cannot change after creation, so it is safe to pass to many appenders and threads.
  4. Logger loops over its appenders and calls `append(record)` on each. Each call sits in a try-catch, so a broken file appender cannot crash the order flow.
  5. `AbstractAppender.append` checks the appender's own level, asks its `Formatter` for a line of text, and calls `write(line)`. This is the template method: the parent fixes the steps and leaves one step to the child.
  6. `ConsoleAppender.write` prints the line and `FileAppender.write` adds it to a buffered file. The same record appears in plain text on the console and as JSON in the file.

Patterns used

  • Template method: AbstractAppender.append does the level check and formatting, then calls the abstract write(line).. Every appender needs the same steps. Writing them once means a new DatabaseAppender only implements write, and it cannot forget the level check.
  • Strategy: Formatter, with PatternFormatter and JsonFormatter, chosen per appender.. The format does not depend on the target. Any appender can use any formatter, so 3 targets and 3 formats need 6 classes, not 9 subclasses for every combination.
  • Factory with a cache: LoggerFactory.getLogger(name).. Callers ask for a logger by name and get a shared instance with the right level and appenders. They never call `new Logger` or wire appenders themselves.
  • Singleton, used with care: One LoggerFactory for the whole app, created at startup.. There should be one set of loggers and open files per process. Prefer creating it once in main and passing it in, because a static global makes tests share state.

Key code

Levels, the record and the appender template

public enum LogLevel {
    DEBUG, INFO, WARN, ERROR;
    public boolean atLeast(LogLevel other) { return compareTo(other) >= 0; }
}

public record LogRecord(Instant time, LogLevel level, String logger,
                        String thread, String message, Throwable error) { }

public interface Appender { void append(LogRecord record); }

public interface Formatter { String format(LogRecord record); }

/** Template method: the checks and formatting live here once; subclasses only write. */
public abstract class AbstractAppender implements Appender {
    private final Formatter formatter;
    private final LogLevel minLevel;

    protected AbstractAppender(Formatter formatter, LogLevel minLevel) {
        this.formatter = formatter;
        this.minLevel = minLevel;
    }

    @Override
    public final void append(LogRecord r) {     // final: subclasses cannot skip the steps
        if (!r.level().atLeast(minLevel)) return;
        write(formatter.format(r));
    }

    protected abstract void write(String line);
}

public final class ConsoleAppender extends AbstractAppender {
    public ConsoleAppender(Formatter f, LogLevel min) { super(f, min); }

    @Override
    protected synchronized void write(String line) { System.out.println(line); }
}

Logger and LoggerFactory

public final class Logger {
    private final String name;
    private volatile LogLevel minLevel;            // can be changed at runtime
    private final List<Appender> appenders;       // read on every call, changed rarely

    Logger(String name, LogLevel minLevel, List<Appender> appenders) {
        this.name = name;
        this.minLevel = minLevel;
        this.appenders = new CopyOnWriteArrayList<>(appenders);
    }

    public void info(String msg) { log(LogLevel.INFO, msg, null); }
    public void error(String msg, Throwable t) { log(LogLevel.ERROR, msg, t); }
    public void setLevel(LogLevel level) { this.minLevel = level; }

    public void log(LogLevel level, String msg, Throwable t) {
        if (!level.atLeast(minLevel)) return;      // cheap check first: most DEBUG lines stop here
        LogRecord r = new LogRecord(Instant.now(), level, name, Thread.currentThread().getName(), msg, t);
        for (Appender a : appenders) {
            try {
                a.append(r);
            } catch (RuntimeException e) {
                System.err.println("Appender failed: " + e);   // logging must never crash the app
            }
        }
    }
}

public final class LoggerFactory {
    private final ConcurrentHashMap<String, Logger> loggers = new ConcurrentHashMap<>();
    private final LogLevel defaultLevel;
    private final List<Appender> appenders;

    public LoggerFactory(LogLevel defaultLevel, List<Appender> appenders) {
        this.defaultLevel = defaultLevel;
        this.appenders = List.copyOf(appenders);
    }

    public Logger getLogger(String name) {
        return loggers.computeIfAbsent(name, n -> new Logger(n, defaultLevel, appenders));
    }
}

Deep dives

Writing without slowing the app

Weak: Each log call opens the file, writes a line and closes it, on the caller's thread. A slow disk now slows every order. Two threads writing at once can also mix their lines.

Good: Keep the file open with a BufferedWriter and make write synchronized. Lines no longer mix and each write costs less. But business threads still wait whenever the disk is slow.

Great: An AsyncAppender that wraps another appender. Callers put the record on a bounded ArrayBlockingQueue and return at once, and one background thread writes the records. When the queue is full, drop DEBUG and INFO records and count the drops, instead of blocking. A shutdown hook empties the queue so the last lines are not lost.

Trade-off: Async logging can lose the newest lines if the process crashes. Some teams keep ERROR synchronous for that reason. Say which records you are willing to lose.

Routing messages by level

Weak: An if ladder in Logger: ERROR goes to the file and the console, INFO goes only to the console. Every new rule edits Logger.

Good: Chain of responsibility: a DebugHandler passes to an InfoHandler, which passes to an ErrorHandler, and each writes what it owns. It is a common interview answer and shows the pattern. But the chain order becomes a hidden rule, and one target per level is hard to change.

Great: Two level checks and a list. Logger has a minimum level and drops anything below it. Each appender also has its own minimum level, and configuration decides which appenders a logger has. This is how Log4j and Logback work. It handles DEBUG on the console and WARN in the file with no new classes.

Trade-off: The chain teaches a pattern; the level-plus-list design is simpler to configure. If the interviewer asks for chain of responsibility, build it, then explain what real libraries do.

One logging setup per app: singleton or not?

Weak: A static Logger.getInstance() with the file path written inside it. Tests write to the real file, and you cannot have two configurations.

Good: A thread-safe singleton LoggerFactory that holds the configuration and the loggers. There is one place to configure. But it is still a hidden global, and tests that change a level affect each other.

Great: Create one LoggerFactory in main from configuration, and pass it (or its loggers) to the classes that need them. Tests create their own factory with an in-memory appender and check the output. A static helper can still exist for convenience, backed by that one instance.

Trade-off: Passing a factory around adds constructor parameters. Most real code accepts a static entry point for logging only. Say that you know the cost.

Now add one more feature

  • Now rotate the log file daily or at 100 MB. Add a `RollingFileAppender` that extends AbstractAppender and checks the size or date before each write. Put the rule behind a `RolloverPolicy` interface. Logger does not change.
  • Now send ERROR logs to Slack. Add a `SlackAppender` with a minimum level of ERROR. Wrap it in an AsyncAppender so a slow network does not block callers.
  • Now add a request id to every line. Add a per-thread context map, often called MDC (mapped diagnostic context). Logger copies it into each LogRecord and formatters print it. Thread pools reuse threads, so clear the map at the end of each request.
  • Now let operations change a logger's level at runtime. Keep `minLevel` volatile (every thread sees the new value on its next read) and expose `setLevel` through an admin endpoint. Child loggers such as `order.payment` can take their level from `order` by name.

What each level is expected to show

SDE-1: Logger, LogLevel, LogRecord, an Appender interface with console and file, and a Formatter. Level filtering works. Knows that targets and formats should be separate so they can be combined.

SDE-2: All of SDE-1, plus the template method in a base appender, thread-safe writes, and an async appender with a bounded queue and a drop rule. Has a clear view on singletons and adds per-request context for tracing.

Common mistakes

  • Putting file writing, formatting and level checks in one Logger class.
  • Creating ConsoleJsonAppender, ConsoleTextAppender, FileJsonAppender and so on, instead of separating target and format.
  • Building the message string before checking the level, which wastes work for every dropped DEBUG line.
  • Letting an exception from an appender reach the business code.
  • Using an unbounded async queue that fills memory when the disk is slow.
  • Forgetting to flush and close the file on shutdown, so the last lines are lost.

Lessons behind this answer