Re: Should we test SHOULD?

On 23/09/2013 13:10 , Tobie Langel wrote:
> On Monday, September 23, 2013 at 12:31 PM, Robin Berjon wrote:
>> On 23/09/2013 11:48 , James Graham wrote:
>>> Therefore I conclude that we should take option 1); simply
>>> consider "should" level conditions in the spec as untestable (for
>>> us) as other implementation-specific requirements on UI. If you
>>> really want to call out these somehow, I would be happy for
>>> people to write should.txt files describing all the should level
>>> conditions to guide people writing implementation-specific
>>> tests.
>>
>> I would opt for a "modified (1)". By default, don't test SHOULD
>> (they normally aren't testable) but allow people to use their
>> better judgement and include tests for SHOULD in cases where they
>> feel the specification was exceedingly cautious.
>
> Out of curiosity, what's your judgement re the linked pull
> request[1]? Should these reqs be tested?

No, I don't think that the requirements in that section ought to be 
tested, they relate to UI/QoI and don't present interoperability issues.

The situation is different for another SHOULD in that spec:

"""
When the capture attribute is specified, the user agent SHOULD invoke a 
file picker of the specific capture control type.
"""

That statement is more or less requiring the UA to do "something" with 
the fact that capture has been specified. If you don't test that, you're 
not testing anything. (Furthermore, the requirement isn't very clear — 
it should relate to the form control's activation in some way; but 
that's an orthogonal issue.)

-- 
Robin Berjon - http://berjon.com/ - @robinberjon

Received on Monday, 23 September 2013 11:18:14 UTC