Re: Should we test SHOULD?

On Monday, September 23, 2013 at 12:31 PM, Robin Berjon wrote:
> On 23/09/2013 11:48 , James Graham wrote:
> > Therefore I conclude that we should take option 1); simply consider
> > "should" level conditions in the spec as untestable (for us) as other
> > implementation-specific requirements on UI. If you really want to call
> > out these somehow, I would be happy for people to write should.txt files
> > describing all the should level conditions to guide people writing
> > implementation-specific tests.
> 
> I would opt for a "modified (1)". By default, don't test SHOULD (they 
> normally aren't testable) but allow people to use their better judgement 
> and include tests for SHOULD in cases where they feel the specification 
> was exceedingly cautious.
> 
> The issue of devices not being able to support some functionality is 
> orthogonal, and should really not be handled by use of SHOULD in 
> specifications. For instance, an eInk device won't render colour, but it 
> would not IMHO be a good use of anyone's time to put all mention of 
> colour behind SHOULD in any CSS module that relates to it.

It would be interesting to hear the CSS WG's perspective on the matter. Crude stats over the csswg-test repository report the following:

    $ grep "flags" -r ./ | grep "should" | wc -l
    28
    $ grep "flags" -r ./ | grep "may" | wc -l
    214



i.e. there's roughly 30 tests that are marked as testing SHOULD and over 200 that are marked as testing MAY.

Peter, Rebecca, fantasai, others, any thoughts on how to best handle SHOULD/MAY requirements in specs? For ref, the thread starts here:

http://lists.w3.org/Archives/Public/public-test-infra/2013JulSep/0240.html 

Thanks,

--tobie

Received on Tuesday, 24 September 2013 10:02:52 UTC