Flapping Head

I’m using virtualenv and have some python scripts as part of a django project. The scripts need to be run by cron, but have to use the virtualenv.

There’s a few solutions around, but I went with this shell script which changes to the directory it’s in, activates the virtualenv and runs python for the given script with args.

The script looks like:

#!/bin/bash

# Runs python script within virtualenv
# used by cron

cd `dirname $0`
source env/bin/activate
python "${@:1}"

chmod it so it’s executable.

I run it from cron like:

/srv/www/myproject/runscript scripts/test.py arg1 arg2

The test.py script is at /srv/www/myproject/scripts/test.py – i.e. relative to the runscript script.

This is largely a note-to-self, but I hope it helps you, too.

While integrating an xml api in to a django project, I needed to post some xml to a site. I was using urllib2.urlopen but because my xml contained a non-ascii character (a £ pound sign) I was getting dreaded encoding errors like:

'ascii' codec can't encode character u'\xa3' in position 359: ordinal not in range(128)

After some messing, I came up with this:

input_xml = render_to_string('integration/checkout.xml', {'booking': booking})
req = urllib2.Request(url, input_xml.encode('utf-8'), {'Content-type': 'text/xml; charset=utf-8'})
response = urllib2.urlopen(req)

The key part is: input_xml.encode('utf-8'). The unicode string needs to be encoded to utf-8.

Logging to syslog in Python

I was trying to use the standard Python logging module to write messages to syslog. The logging module has a SysLogHandler class which can log to a local or remote syslog daemon.

With no host specified, SysLogHandler uses localhost which is what I wanted. I tried to use SysLogHandler, but it just wouldn’t work. There was no error when I called the logging methods, but my messages didn’t show up in /var/log/syslog.

syslog module works

Python also has a standard syslog module. I tried it and it worked fine; my messages were written to the syslog file.

For example:

import syslog
syslog.syslog('test')

syslogd isn’t listening

After running Wireshark I found the SysLogHandler was correctly sending a UDP packet to localhost on port 514. I could also see there was an ICMP response indicating the UDP packet was not received on that port. syslog wasn’t listening!

Use /dev/log

Instead of sending to localhost, I wanted SysLogHandler to pass the message to syslog on the local machine in the same way the syslog Python module was doing.

The solution is to pass /dev/log as the address parameter to SysLogHandler. It’s not well documented, but it works.

For example:

import logging
from logging.handlers import SysLogHandler

logger = logging.getLogger()
logger.setLevel(logging.INFO)
syslog = SysLogHandler(address='/dev/log')
formatter = logging.Formatter('%(name)s: %(levelname)s %(message)s')
syslog.setFormatter(formatter)
logger.addHandler(syslog)

Easy when you know how.