Re: How many browsers matter Re: Ringmark is now open source

Hi Joe,

On Apr 12, 2012, at 02:28 , Joe Cincotta wrote:
> This is my first post on this group, so apologies for rushing in like a bull in a china shop...

No need to apologise, that's pretty much par for the course.

> If you look at the rings when you run the benchmark and see grey areas it doesn't really help you - it would make more sense to just remove the 'ring' part of the rendering on the benchmark and create a text list with commonly agreed names for each feature that is being tested in the suite and just render a green tick or red cross.

Am I reading you correctly that your issue is with the ring-based presentation in the Ringmark implementation? If so that's just an implementation detail and can easily be changed. The core deliverables for this group are its documents and the test suites that go with. The presentation layer of the Ringmark project is more or less incidental here.

(Also note that you can actually get a text-based list from the currently implementation by unfolding.)

> The purpose of this then is of having an agreed set of feature implementation benchmarks for mobile browsers

That's *exactly* what we're doing :)

> To extend on this idea, it would be great to allow browser vendors to then publish the results of the benchmarks as part of their release process (even as part of their nightly build process) to a central repository so that developers could refer to a central W3C location and search it without needing the browser OR the hardware. The database of test results would provide clarity on feature compatibility for specific browser versions - possibly even on a per device basis.

You're in luck today! This is being worked on:

    http://w3c-test.org/framework/

It's still very much work in progress but it's already being used, and becoming more useful. There's a more user-friendly UI being added (but keep in mind that it's very early work, and quite hackish in places and some parts haven't been released to the production server yet):

    http://w3c-test.org/framework/app/suite

More importantly, with respect to your point, the latter is built atop an HTTP JSON API that would make the sort of distributed test submission possible (it's one of the use cases for it). Documentation for that will be released next week.

Also, the framework makes it possible to create a test suite that points to test cases spread over a bunch of other suites using a simple manifest format. This means that specs can build their own test suites, and we can reference those directly to get an R0 view of the whole thing. And creating your own clustering atop the API ought to be possible (if not necessarily easy at this moment).

> In summary, if you want to cluster features, as a developer it makes sense to do that at your discretion, since your application will be using features in a unique way (do I really need level 1 compatibility if I use a single feature that IS implemented on browsers that are not fully 'ring 1 compliant'?).

Sure, but it helps to get implementations aligned somewhere, rather than the current free-for-all cherry picking.

-- 
Robin Berjon - http://berjon.com/ - @robinberjon

Received on Thursday, 12 April 2012 14:43:30 UTC