Re: Language requirements for server side code in the test framework (was: Review of tests upstreamed by implementors)

On Thu, 21 Mar 2013, Tobie Langel wrote:

> On Thursday, March 21, 2013 at 2:11 PM, James Graham wrote:
>> We also have the problem that many of the tests simply won't run in
>> vendor's systems. Tests that require an extra server to be set up (e.g.
>> websockets tests) are a particular problem, but they are rare. More
>> problematic is that many people can't run tests that depend on Apache+PHP
>> (because they run all the servers on the individual test node and don't
>> have Apache+PHP in that environment). Unless everyone is happy to deploy
>> something as heavyweight as Apache+PHP, we may need to standardise on a
>> diffferent solution for tests that require custom server-side logic. Based
>> on previous discussions, this would likely be a custom Python-based
>> server, with special features for testing (I believe Chrome/WebKit already
>> have something like this?).
>
> So I hop[ing to be able to get rid of Apache/PHP in favor of a much more 
> lightweight system, combined with a convention (a manifest?) to set HTTP 
> headers where necessary.

FWIW "being able to set headers where necessary" makes it sound like your 
imagined system doesn't have anything like the level of control that is 
needed to write many kinds of tests. As well as being able to set headers, 
one needs to be able to produce more or less arbitary responses, with 
request-based variation, and also to store state across requests. For 
comparison, see the documentation of Mozilla's javascript based solution 
[1] (Opera have had more or less all the reequirements described in that 
document, but have historically implemented them on top of Apache + PHP 
instead of a custom solution).

I don't think the most significant property of the system is that it is 
simple to use (although of course making easy things easy is a good 
thing); I think the most significant property is that hard things are 
possible.

> I was tempted with a node.js based solution, because you know, 
> JavaScript is the language of the Web. But if picking something else 
> guarantees these tests are going to be run by implementors, I'm willing 
> to learn COBOL (just kidding).

Past conversations suggest that Python is a blessed language in many of 
the relevant institutions. All of Opera, Mozilla and Google/Chromium 
already have Python-based components of their test system. I think 
internally Microsoft use .NET things, which it seems highly unlikely that 
other people will adopt :)

[1] 
https://developer.mozilla.org/en-US/docs/Httpd.js/HTTP_server_for_unit_tests

Received on Thursday, 21 March 2013 16:33:17 UTC