Re: Test assertions requirement

Le 2016-08-15 18:18, Geoffrey Sneddon a écrit :
> (Bcc'd public-css-testsuite; can we please keep responses on a single
> mailing list, in this case www-style as it's about a WG resolution?)
> 
> In the SF F2F, 2016-05-09 we resolved:
> 
> RESOLVED: testharness.js tests don't need a meta assert (but reftests
> still do)
> 
>> From memory, this was based on a mistaken understanding that 
>> assertions
> were currently required everywhere.
> 
> This seems to contradict all documentation going back almost a decade!
> In 2007 fantasai updated the wiki in
> <https://wiki.csswg.org/test/format?do=diff&rev2%5B0%5D=1184350761&rev2%5B1%5D=1186099916&difftype=sidebyside>
> to state that assertions are optional (generically!).

First, I was not in the SF F2F event.

My understanding is: if a test is simple and basic in such a way that 
glancing at the source code makes easy to figure out, to understand the 
purpose/goal of such test, then the test assertion text is not needed. 
But this is an exception rather than a rule. A clear majority of CSS 
tests are complex, not-obvious-to-figure-out, are testing a specific 
statement of the specification or module (or should be in the first 
place about testing a specific statement of a specification or module or 
claiming to be testing some specific statement). Therefore test assert 
text are often needed, desirable, suitable. And with the multiplication 
of specialized modules, this trend will not diminish with time.



> I've been unable to find any discussion leading to this change, though
> obviously discussion was happening around then (see
> <https://lists.w3.org/Archives/Public/public-css-testsuite/2007Jun/0024.html>
> for example).


I swear I read discussions about test assert text in this mailing list 
before and not too long ago ...

Gérard

> Do we want to revisit the resolution in light of being mistaken as to
> previous policy? Or do we just want to consider the documentation 
> wrong?
> 
> /Geoffrey

Received on Tuesday, 16 August 2016 20:23:00 UTC