Re: Test assertions requirement

On 08/15/2016 03:18 PM, Geoffrey Sneddon wrote:
> (Bcc'd public-css-testsuite; can we please keep responses on a single
> mailing list, in this case www-style as it's about a WG resolution?)
>
> In the SF F2F, 2016-05-09 we resolved:
>
> RESOLVED: testharness.js tests don't need a meta assert (but reftests
> still do)
>
>>From memory, this was based on a mistaken understanding that assertions
> were currently required everywhere.
>
> This seems to contradict all documentation going back almost a decade!
> In 2007 fantasai updated the wiki in
> <https://wiki.csswg.org/test/format?do=diff&rev2%5B0%5D=1184350761&rev2%5B1%5D=1186099916&difftype=sidebyside>
> to state that assertions are optional (generically!).
>
> I've been unable to find any discussion leading to this change, though
> obviously discussion was happening around then (see
> <https://lists.w3.org/Archives/Public/public-css-testsuite/2007Jun/0024.html>
> for example).
>
> Do we want to revisit the resolution in light of being mistaken as to
> previous policy? Or do we just want to consider the documentation wrong?

I think this is just unclear wording in the resolution. The resolution
meant that we still need a standardized way of expressing the assertions
in reftests -- that it's still considered part of the metadata we maintain
for the test suite. It was not intended to make the assertion required.

That said I agree with gtalbot and r12a that we should *recommend* people
document the purpose of their test in its header in the same way we
*recommend* people document the purpose of their function in their code.
Sadly the quality of test asserts is absymal usually. :( But good asserts,
like good docstrings, should be encouraged...

~fantasai

Received on Tuesday, 16 August 2016 21:19:44 UTC