Re: Test Suites Summary

Hi, Patrick-

Patrick Dengler wrote (on 4/22/10 12:38 PM):
> I was reviewing the telecon notes, and I don't think this information
> was captured well so I wanted to confirm that we were all on the same
> page.

Minuting is hard when you're one of the participant in the conversation. 
:) Unfortunately, we were all talking, so nobody was able to spell Chris 
for this bit.


> The SVG 1.1 Test do not test conformance, but rather test some
> elements of the implementability of the spec.  Because of this, we
> are giving the wrong perception to developers. One action taken here
> was to update the SVG Test Page to read :" These tests, and the
> corresponding implementation reports are intended not to certify
> implementations as conforming, but rather the interoperable
> implementability of the specification itself."

I have amplified this in the wiki, adding a new section to explain some 
of the subtleties [1].  We can't stop people from misusing these tests, 
but at least we can supply them with the correct information.


> The other note was that, this being the case, we as the WG need to
> remain vendor neutral. Despite Jeff's effort here, linking from the
> SVG WG to Codedread which states boldly at the top of the page "NOTE:
> These web pages look much better in a modern browser like Firefox,
> Opera or Safari or Chrome. Please consider downloading and using one
> of these fine browsers" as Doug noted by definition is biased and we
> need to remove that link. The action is to remove this link as soon
> as possible.

Chris and I talked after the telcon, and we agreed that that link should 
be removed, because it isn't necessary for the main goal of our tests 
suites: to indicate how interoperably the spec is implemented to judge 
the state of the spec; and to provide tests to implementers for use as 
part of their internal test suites.  Chris removed that link immediately 
[2].


> The primary outcome however of this discussion is that we need to
> provide information to developers about the interoperable set of SVG
> across browsers in a vendor neutral way.  We are at risk that if the
> public relies the SVG Test suite to understand interoperability of
> browser implementations, as that view will be skewed and developers
> are likely to not adopt SVG.   And that this is not the desired
> result.

Right.  Jeff has done good work in updating that test report, but I'm 
sure he would agree that he is working from an incomplete test suite; 
the true goal we are all working for is an interoperable set of SVG 
features, to make SVG as usable as possible by authors.  Ideally, we 
would have a truly representative test suite, which would involve far 
more tests; but realistically, it will take a lot of resources for us to 
build up to that point.  We should make that a serious goal for SVG 
2.0... to have a much more complete and representative test suite, so 
even if it's mistaken for a conformance test suite, it shows an accurate 
picture.


>While I offered to submit the right set of tests to the SVG
> WG from Microsoft,

I am not trying to discourage you from submitting tests... quite the 
opposite.  But let's do that as part of the SVG 2.0 spec, not SVG 1.1 
SE.  That will give us a better opportunity to fine tune the 
specification to meet the needs and expectations of authors.


>it was unanimously agreed that this would take up
> significant time from the WG and this would be "very bad for morale
> to hold up 1.1se while the test suite gets bigger" because it took a
> year to do this for 1.2 Tiny.  I certainly don't want to hold up 2nd
> edition.

I appreciate that.  I am also (and even more) concerned that we not 
delay the work on SVG 2.0 any more than necessary.


>Doug suggested that Microsoft work with the SVG Interest
> Group to redefine the "SVG Torture Tests" to be a more accurately
> balanced set of SVG tests that will leave developers with a clear
> understanding that the vast majority of modules (20/23) will be
> available to them across browser after IE9 ships.

I strongly support this goal, and I think Jeff, as chair of the SVG 
Interest Group, would support it as well.

Obviously, I want IE to ultimately support all of SVG, especially 
including declarative animation, filters, and SVG fonts, but none of the 
browsers supported all of that out of the gate, and there are certain 
aspects of all of those things that will benefit from clarification and 
collaboration with the CSS and HTML WGs.  There are also the new 
features of SVG that no browser supports (like SVG Parameters, whenever 
it gets done) that I want to see taken up.  But step one is to have a 
solid foundation of interoperable features.


> I want to make sure we are all on the same page as this discussion
> was brought up before and I feel like we have a more concrete set of
> actions this time.

Agreed.


[1] http://www.w3.org/Graphics/SVG/WG/wiki/Test_Suite_Overview
[2] 
http://www.w3.org/Graphics/SVG/WG/wiki/index.php?title=Test_Suite_Overview&diff=1853&oldid=1852

Regards-
-Doug Schepers
W3C Team Contact, SVG and WebApps WGs

Received on Thursday, 22 April 2010 22:06:50 UTC