twill + rails
My company has an increasingly large stable of web applications. The biggest one is written in PHP (currently being rewritten in python), but in the last year or so we’ve come to rely more heavily on frameworks, mostly Django and Rails.
Testing Rails apps with twill can be tricky, particularly if they’re RESTful. I haven’t quite figured out why that is, but once my dev refactored his code to make it more RESTful, my tests started failing. Said tests are all written in python, so let’s open up a shell and see what that gets us.
>>> from twill import commands as tw
>>> tw.debug('http', 1)
DEBUG: setting http debugging to level 1
>> tw.go(http://qa.xxxxx.com/xxxxx)
connect: (qa.xxxxx.com, 80)
send: 'GET /xxxxx HTTP/1.1\r\nAccept-Encoding:identity\r\nHost:
qa.xxxxx.com\r\nConnection: close\r\nAccept:
text/html; */*\r\n\r\n'
reply: 'HTTP/1.1 500 Internal Server Error\r\n'
header: Date: Tue, 20 Jan 2008 22:10:13 GMT
header: Server: Mongrel 1.0.1
header: Status: 500 Internal Error
header: Cache-Control: no-cache
header: Content-Type: text/html; charset=utf-8
header: Content-Length: 9742
header: Set-Cookie:
_session_id=23e1a01fb82c5356cda6cb34fadfabea; path=/
header: Connection: close
==> at http://qa.xxxxx.com/xxxxx
'http://qa.xxxxx.com/xxxxx'
Rails routes traffic based on headers, so let’s hit this random header testing site.
In twill, you get this:
Accept: text/html; */*
Accept-Encoding: identity
But hitting it in my browser gets me:
Accept: text/xml, application/xml, application/xhtml+xml, text/html; q=0.9, text/plain; q=0.8, image/png, */*; q=0.5
Accept-Encoding: gzip, deflate
The thing to pay attention to, really, is that q= stuff in the browser’s accept headers.
So! After much digging around, there are two ways to fix it (well, three, where the third is ’submit a patch,’ which I haven’t yet done). To fix this issue in twill itself (which is what I’ve done, even though it means my tests are not particularly portable), open up browser.py and take a look around line 63. It reads:
# Ask for MIME type 'text/html' by preference.
self._browser.addheaders = [("Accept", "text/html; */*")]
What that means is that twill’s default accept header is:
Accept: text/html; */*
Which, according to the http spec, is incorrect. What that header does is give equal preference to two different MIME types, which causes Rails to freak out. It doesn’t know what the browser wants, so it doesn’t send anything at all. (That may be an issue in Rails as well — if it can’t figure it out, why doesn’t it pick something rather than send nothing?)
Anyway, the twill-centric fix is to change that line to:
self._browser.addheaders = [("Accept", "text/html; q=0.9 */*")]
Adding the “q=0.9″ sets the priority for */* at .9, leaving the priority for text/html at 1. Now that the two different MIME types have two different priorities, Rails knows what the browser wants and doesn’t freak out.
If you want to fix it without breaking into twill and also keeping your tests portable, you can add this to your tests:
from twill import commands as tw tw.clear_extra_headers()
That will clear out the bad default accept headers that twill is sending, and will allow your tests to work. Fancy!
Tags: python, rails, twill
No comments yet.