Testing Requirements issues - some proposed solutions

Hello Tav,

 Looking at the issues towards the end of
http://www.w3.org/Graphics/SVG/WG/wiki/SVG2/Testing_Requirements

I have some suggestions for them.

*   We will need a new NS URL for SVGTestCase. 

Yes. The existing one is
http://www.w3.org/2000/02/svg/testsuite/description/
which doesn't point to anything (nor need it point to anything, to be clear).

The reason for using another namespace is to follow our own rule about putting non-SVG things in a different namespace.

Since it doesn't much matter I propose
http://www.w3.org/2012/svgtest/

 *   Do we need to keep the copyright section? The CSS tests do not include a copyright. 

Yes we should have a copyright statement but it can be a link, it need not be a whole paragraph of text.

Similarly we should have a license statement, which the original SVG tests had and the later ones dropped, again it can be mostly a link.

Here is some info that describes the test suite licenses
http://www.w3.org/Consortium/Legal/2008/04-testsuite-copyright.html

Their suggested markup (needs an enclosing element, such as <p> in the html namespace)

Distributed under both the <a href="http://www.w3.org/Consortium/Legal/2008/04-testsuite-license">W3C Test Suite License</a> and the <a href="http://www.w3.org/Consortium/Legal/2008/03-bsd-license">W3C 3-clause BSD License</a>. To contribute to a W3C Test Suite, see the <a href="http://www.w3.org/2004/10/27-testcases">policies and contribution forms</a>.

We could either include all that, or put it in a page somewhere and point to it.

*    The CSS that the CSS test is testing is included in the <head> section where it can easily be found. This idea would be nice for SVG tests that test CSS so the CSS can be displayed along with the test (example). 

Yes. Originally we were writing the tests in one format then stripping out the non-SVG namespace stuff (mainly so Batik didn't complain about DTD validity, as I recall). 

* However we cannot put our CSS in the same place. We need a style section outside the SVGTestCase block with an appropriate 'id'.

Do we? Why? 

*   Do we need to keep the revision displayed in the SVG or can we rely on Mecurial? 

I suggest not.

We originally added it because we sometimes found that a test had changed while a patch had not, so we wanted a visual check that they were the same revision.

Then we found that often, a test would be changed (for example to clarify the description) but the patch was still correct, and needed to be updated. Then this happened so often that we just ignored the in-your-face revision number.

So I suggest we don't display it.

*    Do we need the "Not Approved" warning box? The adopted test directory structure keeps the approved tests separate from unapproved tests. 

And thus, we don't need it because the risk of going through a series of tests and accidentally seeing an unapproved one is gone.

*    CSS has a meta section for "flags" to indicate what features are needed by the renderer for the test. Two flags relevant to SVG are: "svg" and "namespaces". Since SVG is SVG, do we need to include these flags or can we just assume that any SVG renderer will support SVG and namespaces? 

We need the flags because they are used by the build system and because tests can be shared between groups (for example the CSS WG can use tests that we make, for specs that are jointly developed). Similarly the CSS WG makes XHTML tests and they have the xhtml flag despite the fact that all CSS/HTML renderers will understand html. Which helps us, since those tests can not be sent to SVG-only renderers.

We may also need additional flags 

- font if the test requires a specific font to be installed to run the test
- optional if the test is for an optional feature

If we have multiple conformance classes then we may need flags for those. For example, fully color managed user agents vs. minmally color managed on unmanaged ones.

It is not clear to me if a multi namespace XHTML + SVG (or html5 + svg) test can simply indicate that with flags="xhtml svg" or if we need a new "html+svg" flag as well.

Not listed as an issue, but it is an issue:

* We previously had per-test description, instructions for running the test (weirdly named 'operator script') and explicit pass criteria. Those were useful, especially the pass criteria. 

In the old system, our build script extracted that and put it into the test harness so it would be displayed along with the actual SVG test and the reference image.

Where do those go, in the new system? And how do they get displayed, since the test harness is auto generated and won't pull in that information?

Also

* we need a similar, but simpler, template for the reference images.

-- 
 Chris Lilley   Technical Director, Interaction Domain                 
 W3C Graphics Activity Lead, Fonts Activity Lead
 Co-Chair, W3C Hypertext CG
 Member, CSS, WebFonts, SVG Working Groups

Received on Thursday, 8 March 2012 19:27:35 UTC