Re: Soliciting feedback on draft-abarth-url

On Tue, Apr 19, 2011 at 10:26 AM, Julian Reschke <julian.reschke@gmx.de> wrote:
> On 18.04.2011 07:51, Adam Barth wrote:
>> Greetings public-iri,
>>
>> I'm starting to actively edit draft-abarth-url again.  If you have any
>> technical feedback on the document, please let me know.  Particularly
>> useful is feedback in the form of a test case that can be added to
>> this test suite:
>>
>> http://trac.webkit.org/browser/trunk/LayoutTests/fast/url/
>>
>> As an example, the following test suite shows how a number of
>> different sequences of characters are segmented into components:
>>
>>
>> http://trac.webkit.org/export/HEAD/trunk/LayoutTests/fast/url/segments.html
>>
>> Test cases are especially helpful because they allow us to compare the
>> behavior of different user agents and will ensure that the net result
>> of this process is interoperable behavior.
>>
>> You can always find the latest version of the draft at this location:
>>
>> https://github.com/abarth/url-spec/blob/master/drafts/url.xml
>>
>> I'm not soliciting editorial or presentational feedback at this time.
>> If you have editorial or presentational feedback, I'd ask that you
>> wait until we've fleshed out the test suite and core technical content
>> of the document.
>
> Here's an observation: FF4 fails most of these tests, IE9 fails all of them.
> So whatever these tests test is not relevant in practice.

I should have mentioned that the PASS/FAIL judgements of these tests
are set somewhat arbitrarily.  The test exist to probe behavior.  It's
our job to look at the behavior and reach judgements about them.

> As far as I understand, these tests use the URL decomposition features of
> the HTML anchor elements. Last time I tested those (when I looked at the
> HTML spec), I noticed that browsers vary in things like
>
> - how to default the port number
> - whether the returned query part starts with a question mark
> - empty string vs exception
>
> I can see why it would be attractive to reduce the differences, but this
> *really* is an HTML API problem that is only indirectly related with parsing
> references.

There are two layers here.  The first is the underlying processing of
the URL and the second is the APIs we have to probe that underlying
processing.  The two belong in separate specifications.  However, in
order to test the former, we need to use the later.  We need to use
our human judgement to understand which aspects of the behavior we
observe are a result of the former and which are the result of the
later.

> Questions:
>
> 1) For the sake of testing *parsing*, can we simplify the tests so that they
> don't "fail" for cases like port number defaulting? Is this difference even
> relevant in any practical way? Does any JS code out there depend on this?

The PASS/FAIL judgement isn't relevant to our work.

> 2) Is there another way we can test browsers in a more relevant way? Just
> because it's easy to test a specific API doesn't mean that this is what will
> tell us anything of significance.

I'm open to suggestions.  The main requirement is that we can perform
the test in a black-box manner so that we can test both open- and
closed-source implementations.

Adam

Received on Tuesday, 19 April 2011 17:38:31 UTC