Re: Ringmark, Core Mobile, and the Financial Times

On 06/10/2012 11:03 PM, Robert Shilston wrote:
> Dear all,
>
> I've been holding off from posting to the list whilst I've been
> getting a sense of the attitudes of existing members.  However, the
> discussions have been relatively quiet so I thought I'd contribute my
> thoughts ahead of the face to face meeting.  To put these in context,
> let me introduce myself.  I'm the engineering manager of the
> Financial Times' web app, and founder of Assanka which was acquired
> by the FT to form FT Labs.
>
> In short, one of our challenges is to assess the amount of work
> needed to be done to for app.ft.com to support an arbitrary new
> device.  What we've found with many new devices is that the features
> we need might be present, just not behave very well.  So, just as Jet
> from Mozilla said a couple of weeks ago, the browser may claim to
> support wordBreak, but it not actually work.  Examples we've seen of
> this are, in my mind, closer to browser or hardware bugs than whether
> something's supported.  Here's a few:
>
> 1) webkit animations running at just a couple of frames per second
> (Android Browser) 2) local storage quota errors erratically firing
> (Windows 8) 3) application cache manifest not being re-verified
> (iOS) 4) CSS font family fall back list not being used during the
> loading of an @fontface custom font (PlayBook)
>
> I've given an examples above of issues across a range of different
> browsers.  I definitely don't think there's any value in pointing
> blame or comparing one browser as being significantly better or worse
> than others in this topic area - each of these bugs are things that
> would be very challenging to detect in JavaScript, yet cause us great
> amounts of effort to diagnose and work around.
>
> So, what I'm getting to is how should Ringmark assess these issues,
> or how should the group work to improve the ability for everyone to
> build web apps.

The best solution for bugginess is to have high quality public 
testsuites for various features. The various W3C working groups already 
curate tests for their various specs and have guidelines / frameworks 
for writing tests. The main problem is sourcing the tests themselves; so 
far the dominant contributors by far are browser vendors, and even they 
do not yet contribute as much as you might hope due to a history of 
writing tests in ways that that are specific to their particular browser 
or testing setup. This has been improving recently, however.

In an ideal world web developers would also contribute tests, 
particularly when they find bugs that impede their ability to get things 
working. In particular it would be nice if the general behaviour of 
developers on finding a bug was:

1. Write test (using public framework) demonstrating bugs
2. Submit test to W3C
3. Submit bug report to affected browser vendor pointing at public test
4. Look for short-term workaround

I recognise this is more immediate work than the traditional approach of 
skipping straight to step 4, but it dramatically increases the chances 
of the problems actually being fixed, and improves the interoperability 
of the platform for everyone, including the person writing the test next 
time they try to use the feature. It's also not that much work 
(depending on the bug maybe just a few minutes once you know the 
procedure). I encourage anyone who is in the area to consider attending 
"Test the Web Forward" to learn more about this [1].

Quite how this interacts with ringmark I don't know. For various reasons 
ringmark can't possibly run all the tests needed to verify that specs 
are well implemented, both due to its architecture (one documenent, 
javascript tests only, etc.) and the amount of time more comprehensive 
testing would take. It is not really clear what the value proposition of 
ringmark as an incomplete testsuite is compared to having a static list 
of browsers that are believed (based on more comprehensive testsuites) 
to be "good enough" in their implementation of a given feature.

[1] http://testthewebforward.org/

Received on Monday, 11 June 2012 08:40:02 UTC