Re: Testing Requirements issues - some proposed solutions

On Thu, 2012-03-08 at 20:24 +0100, Chris Lilley wrote:
> Hello Tav,
> 
>  Looking at the issues towards the end of
> http://www.w3.org/Graphics/SVG/WG/wiki/SVG2/Testing_Requirements
> 
> I have some suggestions for them.
> 
> *   We will need a new NS URL for SVGTestCase. 
> 
> Yes. The existing one is
> http://www.w3.org/2000/02/svg/testsuite/description/
> which doesn't point to anything (nor need it point to anything, to be clear).
> 
> The reason for using another namespace is to follow our own rule about putting non-SVG things in a different namespace.
> 
> Since it doesn't much matter I propose
> http://www.w3.org/2012/svgtest/

Cameron's proposal negates the need for this (not using HTML <head>). We
should discuss his proposal.

>  *   Do we need to keep the copyright section? The CSS tests do not include a copyright. 
> 
> Yes we should have a copyright statement but it can be a link, it need not be a whole paragraph of text.
> 
> Similarly we should have a license statement, which the original SVG tests had and the later ones dropped, again it can be mostly a link.
> 
> Here is some info that describes the test suite licenses
> http://www.w3.org/Consortium/Legal/2008/04-testsuite-copyright.html
> 
> Their suggested markup (needs an enclosing element, such as <p> in the html namespace)
> 
> Distributed under both the <a href="http://www.w3.org/Consortium/Legal/2008/04-testsuite-license">W3C Test Suite License</a> and the <a href="http://www.w3.org/Consortium/Legal/2008/03-bsd-license">W3C 3-clause BSD License</a>. To contribute to a W3C Test Suite, see the <a href="http://www.w3.org/2004/10/27-testcases">policies and contribution forms</a>.
> 
> We could either include all that, or put it in a page somewhere and point to it.

Lets make it a link to keep the files compact.

> *    The CSS that the CSS test is testing is included in the <head> section where it can easily be found. This idea would be nice for SVG tests that test CSS so the CSS can be displayed along with the test (example). 
> 
> Yes. Originally we were writing the tests in one format then stripping out the non-SVG namespace stuff (mainly so Batik didn't complain about DTD validity, as I recall). 
> 
> * However we cannot put our CSS in the same place. We need a style section outside the SVGTestCase block with an appropriate 'id'.
> 
> Do we? Why? 

In CSS it is in the <head> section which in our case would be hidden by
the "d" namespace.
> 
> *   Do we need to keep the revision displayed in the SVG or can we rely on Mecurial? 
> 
> I suggest not.

Good!

> We originally added it because we sometimes found that a test had changed while a patch had not, so we wanted a visual check that they were the same revision.
> 
> Then we found that often, a test would be changed (for example to clarify the description) but the patch was still correct, and needed to be updated. Then this happened so often that we just ignored the in-your-face revision number.
> 
> So I suggest we don't display it.
> 
> *    Do we need the "Not Approved" warning box? The adopted test directory structure keeps the approved tests separate from unapproved tests. 
> 
> And thus, we don't need it because the risk of going through a series of tests and accidentally seeing an unapproved one is gone.

Good!

> *    CSS has a meta section for "flags" to indicate what features are needed by the renderer for the test. Two flags relevant to SVG are: "svg" and "namespaces". Since SVG is SVG, do we need to include these flags or can we just assume that any SVG renderer will support SVG and namespaces? 
> 
> We need the flags because they are used by the build system and because tests can be shared between groups (for example the CSS WG can use tests that we make, for specs that are jointly developed). Similarly the CSS WG makes XHTML tests and they have the xhtml flag despite the fact that all CSS/HTML renderers will understand html. Which helps us, since those tests can not be sent to SVG-only renderers.
> 
> We may also need additional flags 
> 
> - font if the test requires a specific font to be installed to run the test
> - optional if the test is for an optional feature
> 
> If we have multiple conformance classes then we may need flags for those. For example, fully color managed user agents vs. minmally color managed on unmanaged ones.
> 
> It is not clear to me if a multi namespace XHTML + SVG (or html5 + svg) test can simply indicate that with flags="xhtml svg" or if we need a new "html+svg" flag as well.

OK, we'll keep.

> Not listed as an issue, but it is an issue:
> 
> * We previously had per-test description, instructions for running the test (weirdly named 'operator script') and explicit pass criteria. Those were useful, especially the pass criteria. 
> 
> In the old system, our build script extracted that and put it into the test harness so it would be displayed along with the actual SVG test and the reference image.
> 
> Where do those go, in the new system? And how do they get displayed, since the test harness is auto generated and won't pull in that information?

They go in the <meta name="assert"  ... /> section.

> Also
> 
> * we need a similar, but simpler, template for the reference images.

CSS uses an identical format except without <link>s and <meta>s (except
that the author <link> is kept).

     Tav

Received on Thursday, 8 March 2012 19:52:19 UTC