Self-contained test setup and ports

I am working on making the test setup self-contained so that it can be 
run on automation, or on individual's machines, with only a known set of 
dependencies, most/all of which we can add as submodules to the 
repository. The end goal for an individual writing tests is that you 
will be able to say something like

python serve.py

and get a full environment identical to that used in production. For 
automation you will be able to do something like

import serve
serve.init_servers()

to start all the required servers for a full testrun.

The known dependencies so far are:

1) wptserve, the Python-based webserver being developed to replace 
Apache+PHP for our purposes

2) pywebsockets, the websockets server that we already use on w3c-test.org.

I know that we also have some tests that rely on the Jetty sebsocket 
server; I would prefer that we rewrite these to use pywebsocket rather 
than trying to support the extra infrastructure.

It is necessary to pick the ports that these servers will run on. 
Ideally we could pick the default ports for each scheme and some other 
ports. This has the significant advantage that some parts of the 
platform have different behaviour for default and non-default ports. 
Without this choice of ports we can't test those features fully. However 
it has the significant disadvantage that these ports are considered 
priviledged and as such need root permissions to run. This may not be 
possible in all automation scenarios. They are also likely to conflict 
with existing servers e.g. system-wide Apache/IIS/etc. installs. 
Therefore the simplest alternate option is to make the environment 
creation script (serve.py above) default to picking a random free port. 
I hope this is a good tradeoff.

Another issue that I don't know how to address automatically is setting 
up the subdomains that are required by the testsuite. As best as I can 
tell this requires manual editing of /etc/hosts or the platform 
equivalent. As part of the environement creation process I can check 
that the expected list of subdomains works, but if anyone has a good way 
to automatically create them, I would appreciate hearing about it.

Received on Monday, 23 September 2013 11:40:56 UTC