0x74696d

Logging with Docker

August 3, 2013

I spent a couple of days this week working on a new deployment design using Docker. Obviously Docker is a new project, so the documentation is a bit of a mess and not quite keeping up with progress on the code. You come to expect that on fast-moving open source projects, so we figured no biggie. But the one almost-deal-breaker for us was trying to figure out logging, so I thought I'd do a short write-up on that here.

docker logs

Having a docker logs command available must be what threw me off. Docker captures all the stdout/stderr from the process you're running, and you can get the docker daemon to spit this out with docker logs $CONTAINER_ID. But if the process you're running is a long-running daemon, that's probably less than satisfactory. At first I thought I'd be clever and thought I could periodically run docker logs >> /var/log/myapp.log. Let's try that with a Django app on gunicorn and see what happens.

$ ID=docker run myimage python manage.py run_gunicorn -b 0.0.0.0:8000
$ docker logs $ID
2013-08-03 18:45:05 [1710] [INFO] Starting gunicorn 0.14.2
2013-08-03 18:45:05 [1710] [INFO] Listening at: http://0.0.0.0:8000 (1710)
2013-08-03 18:45:05 [1710] [INFO] Using worker: sync
2013-08-03 18:45:05 [1713] [INFO] Booting worker with pid: 1713
$ # ok, let's do that again
$ docker logs $ID
2013-08-03 18:45:05 [1710] [INFO] Starting gunicorn 0.14.2
2013-08-03 18:45:05 [1710] [INFO] Listening at: http://0.0.0.0:8000 (1710)
2013-08-03 18:45:05 [1710] [INFO] Using worker: sync
2013-08-03 18:45:05 [1713] [INFO] Booting worker with pid: 1713

Uh oh. If I just append the output of docker logs I'll be writing the entire log out to the file each time. That's going to suck after a while. You don't want to use docker logs to write out logs for your process.

mounted volumes

Instead you should bind a volume to the container and write your logs from your process to that mount point. This maps a location in the container's file system to a location on the host. You can then access the logs separately from the running process in your container and use tools like logrotate to handle them.

$ ID=docker run -v /var/myapp/log:/var/log myimage python manage.py
  run_gunicorn -b 0.0.0.0:8000 --log-file=/var/log/gunicorn.log
$ docker logs $ID
$ # nada!
$ tail /var/myapp/log/gunicorn.log
2013-08-03 18:53:11 [1710] [INFO] Starting gunicorn 0.14.2
2013-08-03 18:53:11 [1710] [INFO] Listening at: http://0.0.0.0:8000 (1710)
2013-08-03 18:53:11 [1710] [INFO] Using worker: sync
2013-08-03 18:53:11 [1713] [INFO] Booting worker with pid: 1811

It's fair to note that attaching a volume from the host slightly weakens the LXC security advantages. The contained process can now write outside its container and this is a potential attack vector if the contained process is compromised.

Despite the documentation woes and a few minor headaches like figuring out logging, getting started with Docker was a lot of fun and I think it has a lot of potential. I'll be sharing more Docker stories here going forward.

Follow
Collaborate.
Communicate.
RSS.

© 2013-2019 Timothy Gross

Except where otherwise noted, content on this site is licensed under Creative Common Attribution 3.0 Unported License. The code of this blog and all code content is licensed under the MIT license.