Re: Are self-describing reftests required?

On Thu, Aug 25, 2011 at 7:46 AM, Alan Stearns <stearns@adobe.com> wrote:
> Hey all,
>
> There was some discussion on #whatwg this morning about where to go for
> documentation on writing CSS3 tests that will be accepted. Looking at the
> links on these pages:
>
> http://www.w3.org/Style/CSS/Test/
> http://wiki.csswg.org/test
>
> I see a page on writing reftests, but none of the other pages say much about
> them. The only mention of reftests I could find was the match/mismatch
> metadata. In particular, there's nothing that requires a submitted test be a
> self-describing reftest. I am under the impression that self-describing
> reftests are vastly preferred, if not outright required.
>
> Are reftests a requirement, a strong preference, or something less? If it's
> one of the first two, shouldn't the test suite documentation and test
> writing guidelines mention them?

Self-describing reftests are strongly preferred, because they can be
checked for correctness manually without as much effort.  Reftests in
general are essentially required at this point unless there's
something special about the feature that makes it impossible.

~TJ

Received on Thursday, 25 August 2011 15:39:30 UTC