Re: Proposal for precisely flagging optional specifications

James Hopkins wrote:
> In some parts of the CSS 2.1 specification (and I'm sure in CSS 3, too) 
> some parts of the specification are optional; that is, a vendor "may 
> choose to include the [specification] item because a particular 
> marketplace requires it or because the vendor feels that it enhances the 
> product while another vendor may omit the same item" [RFC2119 
> (http://www.ietf.org/rfc/rfc2119.txt)]. In test cases that test solely 
> optional behavior, the 'may' requirement flag is adequate since it 
> implies that all behavior being tested is optional. However, using the 
> same mechanism for flagging optionality in cases that test an 
> implementation involving, but not restricted to optional behavior, can 
> be a problem since it may be difficult to determine what part is 
> optional and what isn't, from a reviewer's point of view.
> 
> The issue in hand can be applied to writing test cases for 12.5 LISTS 
> since the specification itself is entirely optional [1], based on the 
> fact that vendors are free to choose whether list-items generate marker 
> boxes. For example, a document is created to test how a first-child 
> floated box interacts with the marker box (which is positioned inside 
> the principal box). Since the marker box is rendered as the first inline 
> box in the principal box in this instance, this test translates into the 
> interaction between inline and floated boxes which is documented as 
> REQUIRED in a separate specification. However, the generation of a 
> marker box is optional, and in order to pass this test, marker box 
> generation must be supported.

[responded to this particular case already; it's an error in 2.1]

> The recognized 'may' META flag is unsuitable for flagging this, since 
> the optional behavior in this case is out of scope of the test document 
> itself. Also, the nature of a META element coupled with the 
> restrictiveness of token values (where the 'may' value could be used) 
> for the 'flags' attribute, means that authors have no way of flagging 
> where the option within the specification actually lies. As a result, a 
> reviewer may find it difficult to determine exactly what the optional 
> behavior is, especially if the author has included a correctly-written 
> assertion which should contain a "complete detailed statement expressing 
> what specifically the test is attempting to prove".
> 
> To resolve this issue, I'd like to propose a new 'assumption' META 
> field. The object of this field is to flag any optional behavior and, 
> more importantly, describe what the optional behavior is and where it 
> lies within the specification. For readability, one 'assumption' is 
> allowed per META field, and multiple 'assumption' META declarations are 
> also allowed. For example, when applied to tests based on the optional 
> marker box generation, it would look like:-
> 
> <meta name="assumption" content="Support for marker box generation" />
> 
> Although the 'assumption' META declaration could conceivably be used in 
> every test regardless of whether or not the test relies on optional 
> behavior (since support for any specification could be assumed in 
> advance of testing an implementation), it should be reserved only for 
> tests that rely on optional behavior which cannot be described simply by 
> the META 'flags' declaration.

If the goal of this META is to aid the test's reviewer, then I think just
including an "if" clause in the assert should be sufficient.
   "If the UA does this, then it must do that."

But if the goal of the META is to aid the test runner, then probably the
best thing to do is to describe the test such that it passes in either
condition. An example of this would be Gérard's tests for the default
attribute selectors:
   http://test.csswg.org/source/contributors/gtalbot/submitted/default-attribute-selector-001.xhtml
which are designed to pass both if the UA matches against default attribute
values and if the UA does not match against default attribute values,
but to fail if the UA does something else which is not allowed.

If you've got some cases where you can't figure out how to do this, post
here and we'll try to help you come up with ideas.

~fantasai

Received on Wednesday, 2 December 2009 21:50:51 UTC