An update about Twiggy, my new Pythonic logger.

What’s New

Yesterday I released a new version 0.4.1 of Twiggy. This release adds full test coverage (over 1000 lines, nearly twice the lines of actual code). I’ve fixed a number of important bugs in the process, so you’re encouraged to upgrade.

The features system is currently deprecated, pending a reimplementation in version 0.5. Features are currently global (shared by all log instances); they really should be per-object so libraries can use them without stepping on each other. Expect some clever metaprogramming voodoo to make this work while keeping things running fast.

What’s Next

Here’s a little preview of what you can expect over the next few weeks:

Be the best, steal from the rest

I’ll be adding support for context fields, a feature inspired by Logbook’s stacks. This allows an application to add fields to all log messages on a per-thread or per-process basis.

>>> from twiggy import * >>> quickSetup() >>> log.process(x=42) >>> log.thread(y=100) >>> log.debug(‘yo’) DEBUG:x=42:y=100:yo >>> def doit(): … log.debug(‘no y’) … log.thread(y=999) … log.debug(‘different y’) … >>> import threading >>> t = threading.Thread(target=doit) >>> t.start(); t.join() DEBUG:x=42:no y DEBUG:x=42:y=999:different y

This is a killer feature for logging/debugging in webapps. One often wants to inject the request ID into all messages, including libraries that don’t know/care that they’re running on the web. There’ll be methods for clearing these contexts, as well as context managers to use with the with: statement.

Stdlib compatibility layer

0.5 will improve compatibility with the standard library’s logging package. This compatiblity will be two-way. You’ll be able to:

• configure twiggy to use stdlib logging as an output backend
• inject an API shim that emulates basic logging functionality

The later requires some explanation. 90-plus percent of the logging code I’ve ever seen only uses the most basic functionality: creating loggers, logging messages and capturing tracebacks. For such code, it should be possible to do:

from twiggy import logging_compat as logging log = logging.getLogger(“oldcode”) log.info(“Shh, don’t tell”)

Even better, twiggy will provide a logging_compat.hijack() method to inject itself into sys.modules so that no modification to old code is needed at all.

I don’t expect this compatibility layer to work for everyone – notably, custom handlers won’t be supported (the underlying models are just too different), but this should ease the transition pain for many people.

Indentation

Also planned for 0.5 is support for user-defined counters. This feature is still taking shape, but it’ll look something like:

>>> def deep(): … with log.increment(‘depth’): … log.info(“it’s dark”) … abyss() … log.warning(“coming back up”) … >>> def abyss(): … with log.increment(‘depth’): … log.info(“it’s cold”) … >>> deep() INFO:depth=1:it’s dark INFO:depth=2:it’s cold WARNING:depth=1:coming back up

Outputs will be able to transform the depth field into useful visual formatting – for example, by using indentation to group lines together in a console app, or by setting a CSS class in HTML. Hell yeah, structured logging.

Etc.

Other forthcoming changes include: a port to Python 3, PEP-8 compliance, rewriting the features system, support for the warnings module and various minor enhancements. I’ll continue to support Python 2.7 using 3to2

N+1

I should probably stop there, but I’m excited by what’s further down the road. That includes:

• lazy logging: an output backend that groups messages together by a key, and only outputs them if some condition is met. For example, capture messages by request ID, and output all of them together if any one message is ERROR or higher.
• cluster logging: Twiggy will support easily settting up a master logging daemon to receive messages from multiple processes on a machine or across your cluster.
• unittest support: stuff the expected log output in your test docstring, apply a decorator, and Twiggy will add additional asserts to ensure your logs come out right.
• backends, backends, backends: email, HTTP, SQL, CouchDB, syslog, NT event log… Maybe even backends that open tickets in your bug tracker or stream live logs to your browser. Yeah.

What do you want?

Now is your opportunity to let me know what you want in a logger. Got a feature I haven’t thought of? Crazy idea? Think I should implement your favorite backend sooner? Tell me in the comments below.