Re: https/wss testing in web-platform-tests

On 23/09/14 19:05, James Graham wrote:
> This is, I think, the next important thing to work on since it is
> required for testing a large number of things. It's also rather
> complicated since real CAs can't let us do the things that would be
> needed to let everyone run their own web-platform.test domain.

So I have a working version of this locally. The next question is how to
deal with the fact that in some setups running https tests will not be
possible, but in other setups it will.

Practically the main problem is that server ports are usually specified
like:

var server_ssl = "https://{{host}}:{{ports[https][0]}}/"

If there is no https server running this currently throws because the
https server doesn't have any associated ports. It would obviously be
possible to make it do something different here, like have the template
return the empty string, leading to nonsense like

var ssl_server = "https://web-platform.test:/"

after substitution. This would work OK in the sense that you could
design tests around the assumption that this would happen, but it seems
pretty unfriendly. It would also be possible to significantly increase
the complexity of the template language and have conditional constructs
like:

{{#if len(ports[https])}}
//ssl is enabled
var server_ssl = "https://{{host}}:{{ports[https][0]}}/"
{{#else}}
var ssl_server = null;
{{#endif}}

An alternate strategy would be to require that all https tests and
associated templating is put in files that aren't used in the non-https
case. These top-level test files would have some metadata to indicate
that they require https and wouldn't even be loaded in the non-https
case e.g. <meta name=https content=true>.

I'm not sure which option's better for test authors. The first allows
more centralisation, but requires that you start all ssl-using tests
with something like

assert_not_equals(ssl_server, null);

The second requires that you remember to put ssl metadata into the top
of every test file and requires a strict separation between things that
will will ssl and things that will not (such a strict separation doesn't
currently exist).

Is the a better option that I'm missing? Do people have a preferred
solution?

Received on Thursday, 9 October 2014 14:17:02 UTC