RE: [widgets] conformance requirements review

Hi Dom,

Thanks a lot for the very useful tool.

Just FYI I created a tool to check the BONDI specs under:
http://bondi01.obe.access-company.com/
It automatically checks each new release of the BONDI specs directly from the SVN repository.

One of the recent outputs e.g. http://bondi01.obe.access-company.com/1_0_3506_87/, reveals several issues (BTW: they will be fixed next week).
The tool is to check also for some semantic inconsistencies in the specs and is actually a set of checkers.

The  checkers (as of today) are as follows:
Statistics
- information how many interfaces, modules, methods, attributes and constants are defined within BONDI 1.0 widls.

Words in documentation
- occurrences of each word. Allows for finding incorrectly spelled words.

Automatic checkers:
Versions checker
- checks whether widls are correctly marked as version 1.0

Types checker
- checks types that are defined and used within BONDI 1.0

Callback Extended Attribute Checker
- checks whether Web IDL callback extended attribute is used as designed for BONDI 1.0 APIs

Error constants checker
- checks whether error constants are properly defined and used within BONDI 1.0 widls

Methods, their return values and parameters
- checks for inconsistencies between methods' Web IDL and corresponding documentation within widls

Visual checkers. Human compiler required:

Method parameters checker: Sorted by parameter name
- enables spotting errors in method parameters' name and type patterns. It would be nice to have the same name for the parameters of the same type and purpose within various methods.

Method parameters checker: Sorted by parameter type
- as above, just sorted differently

Attributes checker: Sorted by attribute name
- enables spotting errors in attributes' name and type patterns. Similar remark as for parameters checker.

Attributes checker: Sorted by attribute type
- as above, just sorted differently

Constants checker
- enables spotting inconsistencies in constants' name, type and value patterns. Similar remark as for parameters checker.

Interface hierarchy checker
- focuses on the interface hierarchy. It enables checking whether the Web IDL interface hierarchy in widls matches the intended one, e.g. from BONDI API Design Patterns (http://bondi.omtp.org/1.0/apis/BONDI_Interface_Patterns_v1.0.html#webidl).

The automatic checkers may be used against any regressions within widls, i.e. the problems are found by the script.
Other checkers rely solely on human bug spotting so any checks have to be repeated with each new widl release.

I assume that further work on the tool set will enable e.g. automatic test case and conformance report generation, once such tool matures.
In case of the BONDI specification checker one of the major factors that enabled such a tool was introduction of the format in which the specs are written, combination of Web IDL and doxygen syntaxes.
I hope that a format could be promptly adopted (probably with modification or simply from scratch) in the DAP (on CC) to facilitate the further work there.

Thanks.

Kind regards,
Marcin


Marcin Hanclik
ACCESS Systems Germany GmbH
Tel: +49-208-8290-6452  |  Fax: +49-208-8290-6465
Mobile: +49-163-8290-646
E-Mail: marcin.hanclik@access-company.com
-----Original Message-----
From: public-webapps-request@w3.org [mailto:public-webapps-request@w3.org] On Behalf Of Dominique Hazael-Massieux
Sent: Wednesday, June 17, 2009 10:04 AM
To: public-webapps@w3.org
Subject: [widgets] conformance requirements review

Hi,

I wrote a simple XSLT to extract the conformance requirements from the
Widgets spec [1], with the following output:
http://www.w3.org/2005/08/online_xslt/xslt?xslfile=http%3A%2F%
2Fdev.w3.org%2F2006%2Fwaf%2Fwidgets%2Ftests%
2FextractTestAssertions.xsl&xmlfile=http%3A%2F%2Fcgi.w3.org%2Fcgi-bin%
2Ftidy-if%3FdocAddr%3Dhttp%253A%252F%252Fwww.w3.org%252FTR%252Fwidgets%
252F

Based on these, here is a review of the Widgets spec based on
conformance requirements analysis:
 * ideally, only the classes of products would appear as subjects of the
conformance requirements; e.g. "A folder may contain zero or more file
entries or zero or more folders" would be rephrased as "A valid widget
package may contain folders with zero or more file entries or zero or
more folders"; this would have two benefits: simplify the analysis of
conformance requirements for building test suites, and identify possible
ambiguities as to what is affected when the conformance requirements is
not respected; that said, I don't think it is crucial so feel free to
not go through all the conformance requirements if that's too much work
 * similarly, conformance requirements that use the passive voice are
suspect (since they often don't tell to which class they apply)
 * "For sniffing the content type of images formats, a user agents must
use the rule for Identifying the media type of an image" -> this assumes
that the user agent already knows the file it is sniffing is an image;
if that's true, the text should make it clear why, and if not, it should
probably be reworded to say that a user agent must sniff for images
format first
 * rather than say "Reserved file names must be treated as
case-sensitive", I would amend the previous sentence to say "The
reserved file names table, below, contains a *case-sensitive*
list..." (and similarly for folder names)
 * "If [...] the start file is not one that is supported by a particular
user agent, then the CC must inform the author": does that mean that a
CC need to know all the supported formats by all user agents? That seems
a bit excessive - I guess I can see cases where a conformance checker
could be configured to report knowledge on a special user agent, but
that would need to be made explicit, and clearly shouldn't be a must
 * "a widget may attempt to access the feature identified by the feature
element's name attribute" I think the "may" here is not intended as a
conformance requirement, so it probably shouldn't be marked as such
 * there is a spurious empty <em class="ct"></em> element in "A feature
   can<em class="ct"></em> have zero or more <a href="#parameter0"
   title="parameter">parameters</a> associated with it."
 * "The steps for processing a widget package involves nine steps that a
user agent SHOULD follow" ; the "should" appear in upper case in the
source, while other conformance requirements are lower case
 * "a user agent must to decompress" is not correct English
 * "If a user agent encounters an unusable file entry, it must ignore
the file entry:" is ended by a colon, but followed by a new sentence
 * "The algorithm always returns a string, which may be empty": again,
this "may" doesn't look like a conformance requirement so shouldn't be
marked as such
 * "that must eventually be presented to the end-user" - I don't think
this is meant as a conformance requirement (e.g. a conformance checker
is a user agent, but will probably never present any of the widget
content to the end-user); I would reword it as "to be presented to the
end user"
 * "an error condition can ask the user agent ignore an object" I don't
think error conditions can ask anything to anyone in general, so I would
rephrase it; I think "ignore" needs a preceding "to" too.

HTH,

Dom

1. http://dev.w3.org/2006/waf/widgets/tests/extractTestAssertions.xsl




________________________________________

Access Systems Germany GmbH
Essener Strasse 5  |  D-46047 Oberhausen
HRB 13548 Amtsgericht Duisburg
Geschaeftsfuehrer: Michel Piquemal, Tomonori Watanabe, Yusuke Kanda

www.access-company.com

CONFIDENTIALITY NOTICE
This e-mail and any attachments hereto may contain information that is privileged or confidential, and is intended for use only by the
individual or entity to which it is addressed. Any disclosure, copying or distribution of the information by anyone else is strictly prohibited.
If you have received this document in error, please notify us promptly by responding to this e-mail. Thank you.

Received on Sunday, 5 July 2009 16:51:11 UTC