Re: QA review of CSS3-UI

On 8/12/03 6:22 PM, "Karl Dubost" <karl@w3.org> wrote:

> 
> 
> 
> Dear CSS WG,
> 
> a little delay due to events and my holidays, for the review of CSS3-UI.
> You will see a lot of "no" but they are conditional, I have sent a
> message on the QA Mailing list for it.
> http://lists.w3.org/Archives/Public/www-qa/2003Aug/0003
> 
> The review is at
> http://www.w3.org/QA/2003/08/css3-ui-qa
> 
> Thanks.

Thank you for your email and your comments.


The first URL mentioned above:

 http://lists.w3.org/Archives/Public/www-qa/2003Aug/0003


> From: Karl Dubost <karl@w3.org>
> Date: Tue, 12 Aug 2003 19:14:17 -0400
> To: www-qa@w3.org
>
> 
> Hi,
> 
> I have made the review of CSS3-UI, which is a modular spec. The
> technology becomes more and more modular with time. It has an influence
> on the way WGs organize their Spec editing. We have moved from a
> monolithic spec to a multiple document spec.

Disagreed. This is not really a change, in that many W3C technologies,
perhaps from the beginning (URL, HTTP, HTML) were designed to be modular,
and work with each other or with other technologies.


> The last big monolithic
> spec have been SMIL and SVG, even if organized in modules.
> 
> Often in this reorganization, you have a master document which defines
> conformance, usage scenarios, etc. and the modules attached to it.
> 
> For example CSS has moved from a 1-spec (css1 and css2) to a multi-spec
> (css3).

Agreed.


> This way of writing specifications raises a very important questions on
> the notion of 1 Recommendation = 1 Technology. It's not anymore the
> case.

Disagreed.  This has already been raised by other technologies that are
based on multiple recommendations, such as Schema [1][2][3] and, more
recently, RDF [4][5][6][7][8][9].

[1] http://www.w3.org/TR/2001/REC-xmlschema-0-20010502/
[2] http://www.w3.org/TR/2001/REC-xmlschema-1-20010502/
[3] http://www.w3.org/TR/2001/REC-xmlschema-2-20010502/
[4] http://www.w3.org/TR/2004/REC-rdf-concepts-20040210/
[5] http://www.w3.org/TR/2004/REC-rdf-mt-20040210/
[6] http://www.w3.org/TR/2004/REC-rdf-primer-20040210/
[7] http://www.w3.org/TR/2004/REC-rdf-schema-20040210/
[8] http://www.w3.org/TR/2004/REC-rdf-syntax-grammar-20040210/
[9] http://www.w3.org/TR/2004/REC-rdf-testcases-20040210/
 

> 1. Should a technology be considered complete when all the bricks are
> ready. So is there a missing concept inside W3C which defines this set
> of bricks?

Agreed.  That is a good question.  For CSS3, each module defines one or more
informative profiles which can be used as such a "set of bricks".  In
addition, there are several normative CSS profiles that have been defined
for particular domains/devices [A][B][C].

[A] http://www.w3.org/TR/2002/CR-css-mobile-20020725
[B] http://www.w3.org/TR/2003/CR-css-tv-20030514
[C] http://www.w3.org/TR/2004/CR-css-print-20040225


> 2. Should a WD (which is a module of this set) be authorized to reach
> the status of Rec (and even Last Call without the other bricks)?

Agreed.  The answer is the same as any other WD which is dependent on other
specifications.


> 2.1 If a module can become a Rec without other elements of the set
> - How do we define depencies?

Agreed.  For the CSS3 Basic UI module, the dependencies are defined in
section 2 Dependencies on other modules.

 http://www.w3.org/TR/css3-ui/#dependencies


> - How do we create usage scenarios?
> etc

Agreed.  Each specification should contain one or more examples
demonstrating to authors how to use the features contained therein.


> AND more important ***How do we apply QA Spec Guidelines?***
> There's a big issue here. We can't for example find the
> conformance section because sometimes it's in a
> document which is not written yet.

Agreed.  Each module should have at least a minimal conformance section.  A
conformance section will be added to the CSS3 Basic UI module.


> 2.2 If we think that a module can NOT become a Rec without others
> elements of the set.
> - How do we create the mechanism in the process document that specify
> this.

Disagreed.  No additional mechanism is necessary, since the presumably the
process document already handles the case of one specifications normatively
depending on another.


> - We have to check that QA Spec Guidelines is working well with
> modular docs
> Notion of table of contents, references, etc.

Agreed.


> Should this be discussed to the chairs mailing-list?

Perhaps.



The second URL mentioned at the top:

 http://www.w3.org/QA/2003/08/css3-ui-qa


> Activities | Technical Reports | Site Index | New Visitors | About W3C | Join
> W3C
> 
> 
> 
> QA Review of Review of CSS3-UI
> 
> Editors: 
> Karl Dubost for the QA Working Group (www-qa@w3.org)
> 
> 
> Status of this document
> 
> This document is a review of CSS3 Basic User Interface Module (W3C Working
> Draft 3 July 2003) against the QA Framework: Specification guidelines. If you
> need to discuss about this review, please, do it on the www-qa mailing-list
> (www-qa@w3.org).

Disagreed.  The CSS3 Basic User Interface Module specifically requests that
comments and discussions (and followups thereof) of the specification take
place on www-style@w3.org.


> This checklist includes all checkpoints from the QA Framework: Specification
> Guidelines [QAF-SPEC] presented in a tabular format.
> 
> QAF-SPEC
> QA Framework: Specification Guidelines, D. Hazaël-Massieux, L. Henderson, L.
> Rosenthal, D. Dimitriadis, K. Gavrylyuk, Eds., W3C Working Draft, January
> 2003, available at http://www.w3.org/QA/WG/qaframe-spec.
> 
> The checkpoints are
> presented by order of their priorities, which makes it an appropriate
> Implementation Conformance Statement for the guidelines.
> 
> General Comments
> 
> A new version of the QA Framework spec guidelines should be released soon and
> will have differences with the one used here. This review is intended to help
> you, not to block you. If needed we could make another review with the new
> version of the QA Framework which will stay in an implementation phase for one
> year.

Agreed.  The editor of this specification will do his best to use this
review to improve the specification as was intended.


> Question about Abstract
> 
> You have written ³The ability to style the appearance of various standard form
> elements in HTML4 and properties to augment or replace some remaining
> stylistic attributes in HTML4.² Could it be misleading for developers that it
> doesn't address XHTML 1.0 or XHTML 1.1 or any future versions of XHTML?

Agreed.  I will add text to the Abstract that references XML, XHTML and
XForms.


> How to read the comments in the table.
> 
> We have decided to make a difference between negative answers when you do not
> comply. CSS3-UI is a module of a bigger set as you have written in the status
> section: ³This document is a draft of one of the "modules" for the upcoming
> CSS3 specification.². One issue is to establish if you do conform to the QA
> Specification Guidelines. It is impossible if we consider only this
> specification. It might be a QA issue. A mail will be sent to www-qa@w3.org
> 
> It raises the question if a WD of a modular set can reach the status of W3C
> Recommendation if other parts are still at another stage.

Disagreed.  As noted previously, the CSS3 Basic UI specification is not the
first (and certainly won't be the last) specification which is modular and
yet normatively depends on other specifications.


> If yes, it means that the QA Spec Guidelines have an issue making it
> impossible to apply.
>
> If no, it means that the CSS WG will have to define the master document which
> defines the missing part before to go to final stage.

Disagreed.  The QA Spec Guidelines should be applied as they are to any
other specification which normatively depends on additional specifications.


> So There will be a lowercase no when it's dependent on the master
> specification and a uppercase NO with red background when it's an issue of
> this module only.

Agreed.  All such "no" or "NO" notes will be responded to.


> Checklist table
> 
> Degree A Conformance
> 
> To be A-conformant with the guidelines, the following Priority 1 checkpoints
> must be fulfilled:
> Nbr     Checkpoint     Yes     No     N/A
> Guideline 1. Define Scope.
> 
> 1.1     
> Include the scope of the specification.
> YES

Agreed.
           
> Guideline 2. Identify what needs to conform and how.
> 
> 2.1     
> Identify all classes of product.
> no: there's no mention of the products which should implement it. You do not
> define also what's an UA. No glossary for this acronym.      

Agreed. A conformance section with definitions and classes of products will
be added.


> 2.2     
> For each class of product, define the conformance requirements.
>       no: See General comments.      
> Guideline 3. Specify conformance policy.

Agreed. A conformance section will be added.


> 3.4     
> If special conformance terms are used, include a definition in the
> specification. 
>       no: See General comments.      

Agreed. A conformance section will be added which defines special
conformance terms.


> 3.5     
> Justify any usage of a dimension of variability by reference to usage
> scenarios and requirements
>       no: See General comments.

Disagreed.  This seems much too heavyweight of a documentation requirement
without much benefit to implementers and/or authors.  Specific dimensions of
variability are documented in individual properties and informative profiles
will be added to document more granular dimensions of variability.  It is
better left up to implementers to judge which dimensions of variability
apply to the usage scenarios particular to their product(s).


> Guideline 4. Address the use of profiles to divide the technology.
> 
> 4.1     
> Indicate whether or not the use of profiles is mandatory for conformance.
>       no: See General comments.

Agreed. An appropriate indication will be added to the conformance section.

       
> Guideline 5. Address the use of modules to divide the technology.
> 
> 5.1     
> If modules are chosen, indicate any mandatory conditions or constraints on
> their usage. 
>       no: See General comments.      

Disagreed.  This requirement does not make sense as a separate requirement
for modules as compared to any other specification which has normative
dependencies on other specifications.


> Guideline 7. Identify the relation between deprecated features and
> conformance. 
> 
> 7.1     
> Identify each deprecated feature.
>       NO: this is a very tricky case. It's not exactly a deprecated feature
but a 
> redefinition of another technology behaviour. How people are supposed to
> implement them. How User Agents should implement HTML 4.01 with CSS3-UI when
> the two compete?     N/A

Disagreed.  First, there are no deprecated features in CSS3 Basic UI.
Second, for any features which may be considered to be "redefined", CSS3
Basic UI simply provides additional details for implementation.


> 7.2     
> For each class of product, specify the degree of support required for each
> deprecated feature and the conformance consequences of the deprecation.
>             N/A 

Agreed.


> Guideline 8. Define discretionary items.
> 
> 8.1     
> State the circumstances for when discretionary items are allowed
>       no: See General comments. it seems that any features could be
> discretionary. It's dependant of your notion of profiles, in this case but you
> haven't defined the profiles.

Agreed.  Profiles will be added.

       
> 8.2     
> For implementation dependencies, address the allowable differences between
> implementations 
> YES

Agreed.

             
> Guideline 9. Allow extensions or NOT!
> 
> 9.1     
> Indicate if the specification is extensible.
>       NO: You should define if the extension are allowed or not. State it.
> 9.2     
> If extensions are allowed, define their scope and constraints.
>             N/A: just because we don't know if the specification is extensible
or 
> not. 
>
> 9.3     
> Prevent extensions from contradicting the specification.
>             N/A: just because we don't know if the specification is extensible
or 
> not.

Agreed.  An Extensions section will be added to the Conformance section.



> Guideline 10. Provide a conformance clause.
> 
> 10.1     
> Include a conformance section.
>       no: See General comments. You didn't provide any conformance clause. You
> have to give one explaining how the different classes of products should
> conform to your specification. Make a reference to the conformance section in
> this spec to the one where it will be defined.      

Agreed.  A Conformance section will be added.


> Guideline 11. Specify how to make conformance claims.
> 
> 11.1     
> Identify and define all conformance designations.
>             N/A: Only because you don't have any conformance sections.

Agreed.  RFC2119 conventions will be used.


> 11.4     
> Impose no restrictions about who can make a claim or where claims can be
> published. 
>             N/A: No conformance section.

Agreed.


> Guideline 13. Clearly identify conformance requirements.
> 
> 13.1     
> Use conformance key words.
>       no: See General comments.      

Agreed.


> 13.2     
> Distinguish normative and informative text.
>       NO: You do not define the section of your documents which are normative.
> Please add after each section, a word to show that this part is normative, or
> that part is informative.      

Agreed.


> 13.3     
> Use consistent terminology.
> YES: It seems correct. Define the terms which are not defined or acronyms.

Agreed.
  

> Degree AA Conformance
> 
> To be AA-conformant with the guidelines, the following Priority 2 checkpoints
> must be fulfilled (in addition to the above Priority 1 checkpoints):
> Nbr     Checkpoint     Yes     No     N/A
> Guideline 1. Define Scope.
> 
> 1.4     
> Provide examples.
> YES           

Agreed.


> Guideline 3. Specify conformance policy.
> 
> 3.1     
> Specify any universal requirements for minimum functionality.
>       no: See General comments.

Agreed.  A Conformance section will be added.


> Guideline 4. Address the use of profiles to divide the technology.
> 
> 4.2     
> If profiles are chosen, define any minimal requirements.
>       no: See General comments. Profiles are chosen but we are unable to
> determine how they will work.

Agreed.  Example profiles will be defined.

     
> 4.3     
> If profiles are chosen, define their relationships and interaction with other
> dimensions of variability.
>             N/A. To be defined.

Agreed.  Such relationship (or lack thereof) will be defined..


> 4.4     
> If profiles are chosen, address rules for profiles.
>             N/A. To be defined.

Agreed.   Implementation suggestions with respect to profiles will be added.


> Guideline 5. Address the use of modules to divide the technology.
> 
> 5.2     
> If modules are chosen, define their relationships and interaction with other
> dimensions of variability.
> YES           
>
> Guideline 6. Address the use of functional levels to divide the technology.
> 
> 6.1     
> If levels are used, define their relationships and interaction with other
> dimensions of variability.
> YES

Agreed.

             
> Guideline 8. Define discretionary items.
> 
> 8.3     
> Indicate any constraints on the number of choices/options that can be
> implemented for discretionary choices
>       no: See General comments.

Disagreed.   Individual features (e.g. properties, selectors) indicate any
choices/options that can be implemented for discretionary choices.


> 8.4     
> Promote consistent handling of discretionary choices.
>       no: See General comments.

Disagreed.   There is consistent handling of discretionary choices in one of
two ways: 1. Most features provide *no* discretionary choices (preferred),
and 2. A few features permit some property values to be handled as other
values.


> Guideline 10. Provide a conformance clause.
> 
> 10.2     
> Make normative reference to specifications on which the current specification
> depends 
> YES

Agreed.

             
> Guideline 13. Clearly identify conformance requirements.
> 
> 13.4     
> Provide a fast way to find conformance information
>       no: See General comments.      

Agreed.  A conformance section will be added.  Notes will be added to the
top of each section and appendix indicating whether the section is normative
or informative.


> Guideline 14. Provide test assertions.
> 
> 14.1     
> Provide test assertions
>       NO: You have a test suite, but you don't provide in this document a way
to 
> know what is testable.      
>
> 14.2     
> Provide a mapping between the specification and the test assertions list
>       NO: You don't have this list.      

Agreed postponed.  A discrete and separate set of testable assertions may be
extracted and listed during test suite development.


> Degree AAA Conformance
> 
> To be AAA-conformant with the guidelines, the following Priority 3 checkpoints
> must be fulfilled (in addition to the above Priority 1 and Priority 2
> checkpoints): 
> Nbr     Checkpoint     Yes     No     N/A
> Guideline 1. Define Scope.
> 
> 1.3     
> Provide Usage Scenarios.
> NO       

Agreed. More examples will be provided.


> Guideline 2. Identify what needs to conform and how.
> 
> 2.3     
> Identify which of the categories of object are specified in the document as a
> whole. 
>       no: See General comments.

Agreed.  Notes will be added to the top of each section and appendix
indicating whether the section is normative or informative.


> Guideline 7. Identify the relation between deprecated features and
> conformance. 
> 
> 7.4     
> Include an explanation for the deprecation.
>       NO: not always. (wrt HTML 4.01)
>
> 7.5     
> Include examples to illustrate how to avoid using deprecated features.
>       NO     

Disagreed.  There are no deprecated features in this specification.  These
checkpoints should show "N/A".


> Guideline 9. Allow extensions or NOT!
> 
> 9.4     
> Define a uniform mechanism to create an extension.
>       NO

Disagreed.  The specification does not define any mechanism to create an
extension, and that should be ok.  CSS2.1 will be referenced for its
mechanisms.


> 9.5     
> Require that extensions be published.
>             N/A. because not defined.
>
> 9.6     
> Require that implementations provide interoperable alternatives to extensions
>             N/A. idem.

Agreed, these checkpoints are N/A.
 

> Guideline 11. Specify how to make conformance claims.
> 
> 11.2     
> Provide specific wording of the claim.
>       no: See General comments.

Agreed.  A Conformance section will be added.


> 11.3     
> Provide a conformance disclaimer.
>       no: See General comments.

Disagreed.  I think this is an excessive guideline that should be dropped.

Note that NO W3C specification (other than QA specifications) provide such a
disclaimer:

http://google.com/search?q=site%3Awww.w3.org+%22conformance+disclaimer%22


> Guideline 12. Publish an Implementation Conformance Statement proforma.
> 
> 12.1     
> Provide an Implementation Conformance Statement proforma.
>       no: See General comments.
> 12.2     
> Require the ICS be completed as part of the conformance claim.
>       no: See General comments.

Agreed postponed.  An implementation report template should be provided as
part of the test suite so that implementers can note explicitly what
features they have implemented when they run their implementation against
the test suite.



Thank you very much for your review.  As you can see (from having read this
far in this response), there are numerous improvements being made to the
CSS3-UI Candidate Recommendation draft as a direct consequence of this
feedback.


Tantek Çelik
editor, CSS3 Basic UI module

Received on Sunday, 11 April 2004 02:49:00 UTC