W3C home > Mailing lists > Public > public-css-testsuite@w3.org > February 2012

Re: Requirements for (level >=3) tests

From: Boris Zbarsky <bzbarsky@MIT.EDU>
Date: Tue, 21 Feb 2012 22:37:32 -0500
Message-ID: <4F4462FC.5050603@mit.edu>
To: Bjoern Hoehrmann <derhoermi@gmx.net>
CC: public-css-testsuite@w3.org
On 2/21/12 10:33 PM, Bjoern Hoehrmann wrote:
> * Boris Zbarsky wrote:
>> If the test is not easy to analyze, it's generally hard to impossible to
>> tell whether the test is demonstrating a bug in the test or a bug in
>> browsers, especially if several browsers agree on their rendering of the
>> test.
>> Note that being easy to analyze is the important thing; good coding
>> practice is only relevant insofar as it aids analysis.
> Assume that the test case exposes a remote code execution vulnerability.

Assume that you have a test case that runs a bunch of code.  The author 
claims it exposes a remote code execution vulnerability, but you can't 
reproduce any memory corruption or crashes or anything like that.

_That_ is closer to the situation I'm talking about, where you can't 
even tell whether the testcase is exposing a bug or not.

Clearly a crash or demonstrated remote code execution is a bug.  Red 
pixels on a screen, on the other hand, may or may not be.  So I think 
your analogy is a bit off.

Received on Wednesday, 22 February 2012 03:38:02 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:13:24 UTC