Python-based test server

I have started work on a python-based test server that I hope we can use 
in order to replace the combination of apache and PHP. I currently have 
some running code but it's at a very early stage, which means that this 
is the ideal time to get feedback about requirements and implementation.

The implementation so far is at [1]. There is a requirements.txt file 
that details some of the requirements that I have kept in mind. To quote:

"""Need a HTTP server with at least the following features:

* Suitable to run on individual test slaves and over the public internet 
(sadly)
* Support plain TCP and SSL servers
* Serves static files with the minimum of configuration
* Allows headers to be overwritten on a per-file and per-directory basis 
(not sure if we need to recurse up the tree?)
* Full customisation of headers sent (e.g. altering or omitting 
"mandatory" headers)
* Simple per-client state (e.g. POST /data/key?value=foo, GET /data/key 
with some sort of transparent session handling and a timeout for keeping 
the value. Mochitest has this, but in a different way, and is single 
user. Need to check how it is used).
* Complex logic in tests (e.g. a different response conditional on 
whether a previous request for the same resource has been made).
"""

So far the server has three modes:

* Normal file handling. This applies to all files that don't fall into 
one of the modes below. By default, files are server with a content type 
based on their extension, although headers can be overridden by 
providing a {filename}.headers file with the format
header-name: header-value
Currently headers are defined per-file only and not per-directory. What 
are the requirements here? In addition, there is a "pipe" facility to 
alter the response properties. Pipes are a set of predefined functions 
that can be activated by providing a "pipe=" field in the query string. 
For example:
GET /foo.html?pipe=status(404)|header(content-type, 
text/plain)|trickle(d1:100:d1)
would send foo.html with a status of 404, the Content-Type set to 
text/plain and with a 1s delay before sending 100 bytes followed by a 1s 
delay and then the rest of the file.

* .asis file handling. These files are literal HTTP responses, sent 
without any alteration.

* .py handling. .py files are expected to define (at least) a single 
function main(request, response). This either directly controls the 
response, by using the response.writer api, which provides both access 
to the underlying socket and a variety of methods for writing specific 
parts of a normal HTTP response, or by setting the status, headers and 
content property of the response object, or by the return value. The 
return value can be a single string containing the response body or a 
two-item tuple containing headers, body, or a three item tuple 
containing status, headers, body. The functions don't run in any kind of 
sandbox (this is technically challenging with python, aiui) and so have 
the ability to do pretty much anything.

At this point the server only runs on a single port and doesn't support 
SSL. Some thought needs to be given to the best way to recreate the 
multiple origins served by w3c-test.org. There is no per-client state 
api, although it should be possible to implement something based on 
cookies "sessions"; I need to examine the requirements for this more 
carefully. There are no custom utility functions available for the .py 
files to use, but it should be possible to fix that somehow.

Any feedback about requirements or implementation details is very welcome.

[1] https://github.com/jgraham/wptserve/tree/initial

Received on Wednesday, 14 August 2013 13:06:03 UTC