RE: Testing and WebMob (was: [W3C Webmob] Call Tomorrow (19th March) Cancelled)

Thanks for the clarification, Dom. I understand that my perception of the status and orientation of the program may be inaccurate. I would like to correct that inaccuracy through clear communication on what the goals and strategy to meet them currently are, but apart from your email below I don't know where to find that information - it doesn't e.g. appear to be on the testing wiki in any "latest plan" state at least.  Thus my perception. And please don’t take that perception as devaluing any of the excellent work and progress in 2013 by those involved - we have definitely moved forward. It's just that we can and will do more, somehow and somewhere.

In W3C we see our value-add in all of this, apart from helping to promote TTWF and getting more people internally actively engaged in producing tests per our priorities (Web&TV, WebMob, and WebRTC use cases primarily), as helping W3C reach that earlier and more ambitious goal. Without a continued organized effort toward that I am skeptical that even the more effective spec-level focus on tests (the short-term goal) will result in a more interoperable and developer-friendly web.

We are willing to lead a task force in W3C to determine how we get there. At the recent Web & TV and Web Payments workshops I had a lot of discussion on specific goals and strategy, including a proposal I made to Jeff re restarting the Web Testing IG to help shepherd this again. Jeff suggested perhaps a joint TF of Web&TV and WebMob. I am OK with that as a place to start, to help drive the test depth and automation goals of our work in those IGs. But I think because its impact is broader, at some point W3C needs to have a dedicated WG or IG that shepherds this work overall for W3C.

Thanks,
Bryan Sullivan | Service Standards | AT&T

-----Original Message-----
From: Dominique Hazael-Massieux [mailto:dom@w3.org] 
Sent: Friday, March 28, 2014 9:23 AM
To: SULLIVAN, BRYAN L
Cc: 'Natasha Rooney'; 'W3C Webmob Public'
Subject: Testing and WebMob (was: [W3C Webmob] Call Tomorrow (19th March) Cancelled)

On mer., 2014-03-19 at 00:12 +0000, SULLIVAN, BRYAN L wrote:
> I’d like to put on the table the concern that the overall testing
> program at W3C has evaporated, or at least retreated into github land
> with no overall plan, organization, or goals.

I don't think that's an accurate characterization of the current
orientation of the testing efforts; before opening the question of what
(if anything) webmob should do in this space, let me summarize what I
understand the current status of testing efforts to be, esp. in
comparison with had been the plans until a few weeks ago.

The work that Tobie accomplished over the past year or so, esp. as a
consequence of the work done in the CoreMob CG (the predecessor of this
group), had two aspects:
* a short-term goal of bringing greater focus and organization to the
ad-hoc work done across groups on testing
* a longer-term goal to develop an ambitious testing program that would
push the envelop of what W3C had been traditionally doing in terms of
testing towards a much more robust kind of testing

The latter had been triggered by various requests from industries to
support their specific needs (esp. TV and Mobile), and was dependent on
getting funds matching the big investment that was required upfront to
enable that type of testing; despite numerous and insistent efforts to
gather such funds, the W3C staff was not able to get a single firm
commitment, let alone commitments matching the scale of the required
investments. As a result, the plans on this front have been set aside;
I'm sure they would be considered again if some funding were to be
offered (but W3C staff will no longer be actively hunting for them).

The former, though, has already had a real big impact on how testing is
being developed in W3C:
* a single repository contains all the tests developed across
browsers-related groups,
* greater consistence around how tests are built, reviewed and
organized,
* a dedicated test runner has been developed to ensure consistency
across runs of these test cases,
* documentations on how to write tests has been collected, published and
made easy to contribute to under the "test the web forward" umbrella
http://testthewebforward.org/

* that same umbrella has continued to host events to create knowledge
and momentum around crowd-sourced test developments

This is already a pretty huge improvement compared to what was before,
and is certainly pretty far from unorganized or aimless.

Now, assuming there is still a need for a more ambitious approach to
testing, assuming there is a particular mobile angle to it (as was
previously discussed in CoreMob), and assuming several WebMob
participants are interested in contributing to its realization, I think
it would make sense to have another task force to work towards that.

If so, it would be useful to get clarity on:
* what would be the mobile-specific goals of the work? we had previously
discussed creating momentum behind more convergence across mobile
browsers, but I think it would be worth looking in more details how that
would work (it could target implementors, developers, standards makers,
…)

* who would be willing to lead such task force and who would be
interested in contributing to it?

(I meant to bring that topic up at our call on Wednesday, but we ran out
of time)

Dom

Received on Friday, 28 March 2014 17:37:29 UTC