- From: Nikos Andronikos <Nikos.Andronikos@cisra.canon.com.au>
- Date: Mon, 24 Oct 2016 03:24:56 +0000
- To: Geoffrey Sneddon <me@gsnedders.com>
- CC: www-svg <www-svg@w3.org>
> On 23 Oct 2016, at 7:53 AM, Geoffrey Sneddon <me@gsnedders.com> wrote: > > Thoughts below… > > On Fri, Oct 21, 2016 at 12:35 AM, Nikos Andronikos > <Nikos.Andronikos@cisra.canon.com.au> wrote: >> Testing >> >> [11]https://github.com/w3c/svgwg/wiki/Testing >> >> [11] https://github.com/w3c/svgwg/wiki/Testing >> >> <AmeliaBR> scribenick: AmeliaBR >> >> Nikos: I created this wiki (link above) to describe the main >> process for how to create tests. >> ... We've mostly agreed to use web-platform-test, but it is >> very web-browser specific, we may need to make adjustments for >> other user agents like Inkscape. >> >> Tav: I think it's straightforward so long as the reference is >> either SVG or PNG. >> >> Nikos: PNG has issues, because then we'd probably need separate >> reference images for each platform. >> >> AmeliaBR: Is that really true? If you're using default system >> fonts, then yes, but for most SVG you should get the same >> rendering on all platforms. >> >> Tav: And for fonts, we can use web fonts to get consistency. > > There's absolutely problems with using PNGs as references: > anti-aliasing varies between platforms, which affects anything that > doesn't fit nicely on the pixel grid (and that's common enough in SVG > tests), and especially affects font rendering given we not only have > anti-aliasing differences but also font hinting differences (and that > typically varies between OS releases, too). > > All the browsers that currently use PNGs for some references want to > move away from them, and changes invalidating several thousand PNGs > are impossible to manually verify, so you just have to assume nothing > bad changed. This is what I was trying to get at during the telcon. I don’t have the practical experience to know all the issues that pop up, but the fact that the browsers are moving/have moved away from maintaining image references is telling. > >> AmeliaBR: I'd be more worried about using SVG as a reference. >> User agents may match the reference rendering on their own >> system, but still not be cross-compatible because of >> lower-level rendering issues. > > Such broken rendering tends to be immediately apparent as soon as you > try and render one page/image, as it almost invariably implies basic > functionality is broken. References, where possible, should also not be using anything more than basic shapes and paths with stroke and fill applied where needed. Of course, the features that are used in the references need to be tested thoroughly themselves - probably with manual tests, though we could compare against HTML/CSS renderings in the browser and InkScape could treat them as manual tests. With a comprehensive enough test suite, errors when rendering references should pop up somewhere, it just might require some investigation to understand where the actual problem is. > >> Tav: Either way, the may concern is that the references aren't >> HTML files, since we can't run them to compare. >> >> Nikos: You could use some other rendering agent to convert >> those to PNG references. >> >> Tav: That adds a new level of complexity. >> >> Nikos: The issue is that web platform tests doesn't give us >> strict control over what gets added. Anyone who's had a good >> reputation with the project gets write access to approve tests. > > It might be practical to add a requirement that the reference of any > SVG is also an SVG (though obviously there's limits to what's > practical/possible: a scripted SVG could of course just add HTML > elements to it dynamically). Yeah, and for fundamental features, I’m undecided whether manual or HTML/CSS references would be better. At TPAC, manual were recommended, but there's the downside they won’t be run that often. > >> Tav: There are many things about it I like. I like being able >> to click and see the reference image. I like having versioning >> control. >> ... One problem is that it is quite a huge repository, and it >> will get even larger as CSS joins. >> >> Nikos: So you don't want to download the whole repo? >> >> Tav: The repo itself is not a big problem, but watching for all >> the issues could be very overwhelming. Having it all in one >> repo seems not very manageable. >> ... I expect it's something that's been discussed internally, >> but for now we'll have to deal with it as best we can. > > Almost all PRs are automatically labelled by a bot and most issues are > manually labelled, so > <https://github.com/w3c/web-platform-tests/labels/svg> for example is > all issues for the SVG spec. > The concern here was the firehose of information you get through Github notification emails. As mentioned on the telcon, one way of mitigating this is to reduce the scope of the repository, e.g. having one repository per technology, but WPT isn’t structured like that, and doing so would cause plenty of it’s own problems. For SVG WG members using WPT, I think there’s some good solutions using email filtering. Github sets out the headers of the notification emails according to a particular scheme: https://help.github.com/articles/about-notification-emails/ The main reason for SVG WG members to follow activity on the repository is to know when there’s tests to review. Since wpt-pr-bot mentions potential reviewers, a filter with the following settings will work to highlight review requests: From: notifications@github.com To: w3c/web-platform-tests CC: mention@noreply.github.com >> Nikos: We could make a fork of the repo, and use it to do all >> the subject-matter discussions, then when we decide on those, >> push them to the main repo for a final approval of the format. >> >> AmeliaBR: I like that idea if we can make it work. Funnel >> things through a 2-stage pull request. > > Note that trying to land huge, large PRs has typically not worked out > well (either for review or for avoiding conflicts). In this case, the PR wouldn’t be any larger - they would be one for one at each step. But in any case, I feel there’s simpler solutions to the problem and setting up a staging repository isn’t the right way to go. > >> Tav: There's already SVG tests there. Mozilla uploaded a lot in >> the past few weeks. >> >> Nikos: Yes, one of the goals of web-platform-tests is to make >> it easy for browsers to dump their internal testing suites, to >> quickly add new tests. >> >> Tav: I like the idea of making it easy to add tests. But that >> doesn't always make it easy to use them. > > Given almost all contributed tests are automated, what makes it hard > to make use of them? I think Tav is referring to format incompatibilities with reftests. Specifically that InkScape needs SVG reftests, anything else will be hard to use. Nikos. The information contained in this email message and any attachments may be confidential and may also be the subject to legal professional privilege. If you are not the intended recipient, any use, interference with, disclosure or copying of this material is unauthorised and prohibited. If you have received this email in error, please immediately advise the sender by return email and delete the information from your system.
Received on Monday, 24 October 2016 03:25:35 UTC