Posting unicode with urllib2

6 January 2011

While integrating an xml api in to a django project, I needed to post some xml to a site. I was using urllib2.urlopen but because my xml contained a non-ascii character (a £ pound sign) I was getting dreaded encoding errors like:

'ascii' codec can't encode character u'\xa3' in position 359: ordinal not in range(128)

After some messing, I came up with this:

input_xml = render_to_string('integration/checkout.xml', {'booking': booking})
req = urllib2.Request(url, input_xml.encode('utf-8'), {'Content-type': 'text/xml; charset=utf-8'})
response = urllib2.urlopen(req)

The key part is: input_xml.encode('utf-8'). The unicode string needs to be encoded to utf-8.

Filed under: Python — Scott @ 2:22 pm