RE: Agenda item for next Conference Call: Test Suite Analysis

Thanks for the quick turnaround.

I am having Sandy review your comments on specific tests where you have questions.

As for the naming convention on tests, I'm not sure you are up to date on the wonderful :) conversations we have had on this subject.

My biggest motivation here is to have developers be able to look at these tests and determine if there exists enough support for SVG across browsers for developers to embrace.  To date, not all browsers implement the entire set, Internet Explorer having the least.  There is no mystery here.

What I don't want is a developer to look at these tests, for example, and think that because Firefox and IE fail enough text- tests, as they don't implement SVG Fonts, that developers can't rely on text in SVG.  

I also have done some asking around as there was a discussion on the conference call last week about the visibility of these tests.  I have learned that these will be extremely visible.

I have proposed the following in the past, and now have a more specific alternate plan (indicated with >>> below):

Microsoft would be happy to contribute many more tests, perhaps at the per-assertion level. This would allow for the specification to be thoroughly tested and conformance and interoperability better expressed.
The SVG Working Group could re-examine the density of these tests and separate out overlapping concepts.
The SVG Working Group could rename these tests as suggested originally in this email. I understand that the CVS system would lose history, but I don't believe this is a strong enough reason not to fix misaligned tests.
>>> The SVG Working Group could modify the conformance template to re-categorize these tests into their appropriate area.

This is the discussion I would like to have.  Again, thanks for the quick turnaround.


-----Original Message-----
From: Cameron McCormack [mailto:cam@mcc.id.au] 
Sent: Wednesday, October 27, 2010 8:51 PM
To: Patrick Dengler
Cc: public-svg-wg@w3.org
Subject: Re: Agenda item for next Conference Call: Test Suite Analysis

Hi Patrick.

Patrick Dengler:
> The following have specific issues identified; they may have been 
> removed from the final test suite interact-order-03-b.svg Test case 
> seems wrong -- removeDefault is called, so clicks will not be able to 
> select.  If you select all, we pass

Agreed.  The preventDefault() call will prevent any of the text within the <a> from being selectable with the mouse.

> pservers-pattern-09-f.svg
> Cannot find case

Not sure what “cannot find case” means.  The test is still un-reviewed, though.  Is the issue that the spec doesn’t say what to do if the xlink:href="" doesn’t reference a <pattern> element?  If so, then I agree that the test isn’t supported by the spec.  The “halt rendering”
behaviour of implnote.html would seem to be required here.

(Ideally we’d define this for <linearGradient> and <radialGradient>,
too.)

> struct-image-12-b.svg
> No pass conditions

I’d classify that as an unfinished test.

> text-align-07-t.svg
> Test needs updated (tiny is updated)

What specifically needs updating?  Assuming that Arial Unicode MS, Georgia, Times New Roman, Times and/or MS Mincho be available might be an issue, but apart from that nothing stood out to me.

> text-fonts-202-t.svg
> Support file not linked

That was raised for text-fonts-203-t.svg recently, too.
http://www.w3.org/mid/op.vkx4azamgeuyw5@mbp-2.local


> types-dom-08-f.svg
> Cannot find case

Details, please.

> The following test cases include SMIL, which either is not integral to 
> the tests, or should be renamed/categorized as animate-

As a general comment, we haven’t had particularly strict rules about naming tests.  The approach I have taken is finding the section for the “main” thing the test is testing, and looking up the corresponding test prefix in http://dev.w3.org/SVG/docs/SVGTestSuite-howto.html, creating one if that section didn’t have one.

A general question, then: is it particularly important how the tests are named?  What if a test is deemed to use two major features equally in combination, how should it be categorised?

> paths-dom-02-f.svg

This doesn’t use SMIL.

> pservers-grad-19-b.svg

This seems to be testing explicitly that the color, stop-color and stop-opacity properties are animatable and have the right effect.  One particular tricky thing it seems to be doing is making sure that animating the color property affects a currentColor value appropriately.

So yes animation is integral to this test, but I would say it is more a test of those properties being animated than the “core animation”
features being tested.

> struct-dom-08-f.svg

I this test I used the <animate> element just to generate timing events.
The test needs to call suspendRedraw() at some time t1, and then call
unsuspendRedraw() at some later point t2 where t1 ≤ t2 ≤ t1 + 10s.  We can’t rely on setTimeout for this, since that’s not an SVG 1.1 feature.

> interact-events-203-t.svg

I think animation is integral in this test, since it is testing that animations within <use> element subtrees work, and that bubbling of events from the shadow tree up to the real DOM can be used to trigger animations.

I’d classify this as a combination animation, event processing and <use> element test.

> interact-pevents-02-t.svg

This one I would classify as an incidental use of animation.  The test is clearly testing pointer events, and not animation per se.

> interact-pevents-07-t.svg

Ditto.

> interact-pevents-201-t.svg

Ditto.  (That is definitely a hard test to operate, incidentally!)

> interact-pevents-202-t.svg

Ditto.

> script-elem-01-b.svg

This is testing that a particular attribute is not animatable.  I would classify that as testing the definition of the attribute more than testing core animation features.

> struct-dom-09-b.svg

This tests SVGSVGElement.getCurrentTime(), which is defined in the struct.html chapter.

> The following test cases include SVG Fonts, which either is not 
> integral to the tests, or should be renamed/categorized as fonts- 
> text-text-04-t.svg

I would say the use of SVG fonts in this test is simply to make visual inspection of the test result easier.  So not integral, agreed.  But the main purpose of the test is testing x="" and y="" on <text>, so categorising it as text-text- makes sense to me.

> text-text-05-t.svg

Ditto.

> text-text-06-t.svg

In this one a font with a ligature is required, so I would say it’s integral to this test.  But again, it’s the x/y attributes primarily being tested, not the ability to support SVG Fonts.

> masking-mask-01-b.svg

Use of SVG Fonts here is useful for the test, to ensure that it is positioned appropriately wrt the mask.  Again, it’s testing masks, not fonts.

> pservers-grad-08-b.svg

Ditto (but for gradients).

> render-elems-06-t.svg

I’d say this is incidental use of SVG Fonts.

> render-elems-07-t.svg

Ditto.

> render-elems-08-t.svg

Ditto.  (And to be honest, this test looks redundant with
render-elems-07-t.svg.)

> text-align-08-b.svg

This seems to be quite similar to text-align-07-t.svg, just with an SVG Font showing triangles rather than the expected glyphs.  So perhaps I would classify this as testing a combination of SVG Fonts and text baseline alignment.


Summarising, I would say that perhaps for a couple of them you could argue they should be named differently, but most of the above seem appropriately named to me.  Renaming means losing CVS history, and losing recorded results against the original name.  I’m not sure it’s worth it.

--
Cameron McCormack ≝ http://mcc.id.au/

Received on Thursday, 28 October 2010 16:49:31 UTC