Deprecated tests

Hi all,

the issue of how to handle deprecated features in the test suite came up at our F2F meeting, and this mail is just a report on how the suite has changed to handle them. I tried to find a simple solution which does not make the test metadata, as well as the underlying software, too complicated. Here is what I ended up doing.

We already had some 'collections' of our tests, using (or slightly misusing…) the 'belongs-to-collection' property. It was used as follows, for example:

    <meta property="belongs-to-collection">should</meta>

referring to a test for a SHOULD feature in the spec. The possible values are 'should', 'may', and 'must' (although 'must' is default and can be omitted). The rationale was that such tests are important and useful, but are not necessary for the purposes of CR exit criteria. To reinforce this, the various test pages (like, e.g. [1]) have a button whereby only the 'must' tests are shown. Such views are used for reporting to the W3C management or the AC when the time comes.

A simple solution for deprecated tests was to add an extra collection, namely 'deprecated'. It is treated like the 'should' and 'may' tests, i.e., hidden from formal reporting, but reading systems are still able to indicate whether implement those features or not. 

This change is now incorporated into the test suite and the underlying reporting software. At the moment there are two tests that have been labeled as deprecated (following the F2F resolution on deprecating rendition:orientation), see at the very end of the FXL test section[2] (make sure to switch the visibility to show all tests!)

This makes it easy to change a test to deprecated: just add (or modify) 

    <meta property="belongs-to-collection">deprecated</meta>

One should also be careful to change the reference to the specification itself to use a fixed (in this case 33) version in the URL; it should refer to the last version that still had a formal specification to the term tested. See the test for orientation set to landscape in [3], especially the dcterms:isReferencedBy property.

Note that I was wondering about the other category of obsolete feature in the spec, namely "obsolete but conforming"[4], on whether something should be done in the test suite about them, too. It would be straightforward to add yet another category (like "obsolete") but the situation is a bit different. An obsolete feature (e.g., font obfuscation) is one which users should really, really not use, but RS-s are still expected to implement it as before, to avoid backward incompatibility problems (whereas deprecated features can just be ignored by RSs). For that reason, I think nothing should change in the test suite for those: RSs should be oblivious to the fact that the feature has been obsoleted but not deprecated

Any comment, change proposals, etc., are welcome

Cheers,

Ivan

P.S. As soon as Hadrien makes a complete list of the to-be-deprecated properties, and the WG accepts that list, I will of course implement the changes in the test suite.


[1] https://w3c.github.io/epub-tests/
[2] https://w3c.github.io/epub-tests/#sec-fixed-layout-data
[3] https://github.com/w3c/epub-tests/blob/main/tests/lay-fxl-orientation-landscape/EPUB/package.opf
[4] https://www.w3.org/TR/epub-34/#sec-obs-conform
----
Ivan Herman, W3C 
Home: http://www.w3.org/People/Ivan/
mobile: +33 6 52 46 00 43





----
Ivan Herman, W3C 
Home: http://www.w3.org/People/Ivan/
mobile: +33 6 52 46 00 43