Summary of Coremob F2F 2012-10-02 and 2012-10-03, Mozilla Offices, London

Hello Coremob

My abject apologies for this summary being so appallingly late, but please find notes from our last Face to Face meeting on the wiki [1] or in monochrome below, also for the benefit of Tracker.

[1] http://www.w3.org/community/coremob/wiki/Meetings/F2F_2012-10-02_and_03/Summary_of_Meeting

You'll note that there are lots of timescales suggested for lots of actions, which have not to my knowledge been actioned. 

Please review the summary and raise questions here on list if there's something you feel you wish to comment on.

Many thanks
Jo

---

Summary of F2F Meeting of Coremob, F2F 2012-10-02 and 2012-10-03, Mozilla Offices, London 

W3C Coremob - Core Mobile Platform Community Group - held a Face to Face meeting, kindly hosted by Mozilla on Tuesday October 2 and Wednesday October 3, 2012.

Agenda: http://www.w3.org/community/coremob/wiki/Meetings/F2F_2012-10-02_and_03/Agenda

Meeting Objectives 

	• Clarify uncertainties that remain over a number of long standing issues
		• What is the relationship with Ringmark?
		• What is the relationship with other initiatives?
		• What are the actual deliverables?
		• What is a realistic timeline?
		• Who can commit to providing resources for that?
	• Bring the Coremob-2012 Spec to the equivalent of Last Call
		• Iron out any remaining issues and actions
		• Discuss what the dividing line between
			• Inclusion in the spec as a definite requirement
			• Inclusion in the spec as something that is fundamental but unrealistic in practice
			• Deferred Altogether to Coremob-2013*
		• Conformance Claims to Coremob-2012
		• Resolve to publish pending the editor enacting the changes agreed
	• Discussion of Test Framework (see document from Tobie)
		• Requirements
		• Front-ends
		• Test Repository
		• Results Repository
	• Discussion of Available Tests
		• From W3C Sources
		• From other Sources
		• How to Cross the Chasm

Meeting Minutes

Meeting Minutes (Day 1): http://lists.w3.org/Archives/Public/public-coremob/2012Oct/att-0030/minutes-2012-10-02.html
Meeting Minutes (Day 2): http://lists.w3.org/Archives/Public/public-coremob/2012Oct/att-0032/minutes-2012-10-03.html

Achievement of Objectives

In the main progress was satisfactory on most fronts, with the exception that it became clear that it would not be possible to advance the Coremob 2012 spec itself to equivalent of Last Call as too many outstanding issues remained, and that there is a dependency on creating a Use Case and Requirements statement.

Each of the deliverables has commitment of effort from members of the Community Group.

Day 1

1. Opening Remarks

Jo Rabin, Chair, welcomed attendees and attendees introduced themselves. Jet Villegas of Mozilla, the meeting hosts, set the scene by making a call to arms about the inherent superiority of the Web platform in its universality, and cautioned the group about the dangers of of badly construed tests and benchmarks as distracting browser vendors from implementing truly useful features. 

To quote Jet on the Web's universality: "I fully expect many of these 'native' applications to be running under javascript emulation on the web for years after the devices they were written for have been recycled."

His presentation is at http://junglecode.net/coremob/coremob.html. 

2. Discussion of what Coremob is here to do, relationship with Ringmark and effective communication of this

Jo suggested that there was significant confusion outside the group as to what the group is trying to do, and what it is not trying to do. He felt that this was due in part to a lack of plainly written words saying what it is. In particular the relationship with Facebook's Ringmark initiative is unclear, with some people thinking that Ringmark is Coremob or that Coremob's role is to rubber stamp Ringmark. Jo pointed out that though significant effort had been given by Facebook, others too needed to make sure that they would contribute, otherwise there is not enough resource to achieve any sensible objectives.

Matt Kelly described the background to Ringmark and why Facebook played a leading role in initiating Coremob. Matt described Facebook's initiative of working with others on 14 Mobile Web app projects. They found it very hard to do and there remain only 3 out of the initial group and that is the reason they decided to co-found Coremob. Ringmark was a separate but parallel effort to highlight things that are important to developers of Mobile Web apps. Matt hoped that Coremob will now take off from there.

There was a lengthy discussion about what, in fact, the group is here to do. Jo elaborated the view earlier articulated by Robert Shilston, Financial Times Labs, that the group exists as a forum for developers to express their needs. Tobie noted that there is little agreement among chipset makes, carriers, OEMs and so on to focus on and that the group could be effective in helping such alignment. Jo noted there is a need for a statement of what is required to allow the Mobile Web to be a reasonable platform for developing Mobile Web applications. There are three parts: A statement of what is required, a framework to run tests in and a set of tests to run in the framework. Jo noted that these parts move at different speeds and that some things that are requirements will never have tests.

It was noted that if the Coremob-2012 spec was "aspirational" then it was not much use to developers today. It was also noted that other groups such as "CanIUse" already provide information in this space and that it would not make sense to duplicate that effort.

Tobie Langel wondered if W3C community tools were adequate for communicating messages clearly and offered to give the coremob.org domain, which he currently owns, to the group. Fantasai (Eleka Etemad) made the highly cogent point that communicating what the group is doing is best demonstrated by the group doing it, not by talking about doing it.

We touched on earlier work by the group concerning a scoping statement for the meaning of "Mobile Web App" - Jennifer Leong of AT&T agreed to continue her work on this under ACTION-66. Dominique Hazaël-Massieux (Dom), W3C, pointed the group at some commentary [1] he'd written on the subject some time ago.

[1] http://people.w3.org/~dom/archives/2010/08/what-is-a-web-application/

There was a short discussion of who in the group participated in other related testing initiatives and other W3C groups. In summary it seems there is a reasonable spread of participants in other groups (see minutes [2] for details).

[2] http://lists.w3.org/Archives/Public/public-coremob/2012Oct/att-0030/minutes-2012-10-02.html#item07

Jo took an action (which did not appear in the list of ACTIONs) to try to summarise the group discussion and circulate a draft of an explanation of the group's purpose. Now raised as ACTION-67, to allow for easier tracking of discussion.

3. Discussion of Coremob 2012

Draft: http://coremob.github.com/coremob-2012/ED-coremob-20120http://dev.w3.org/html5/decision-policy/html5-2014-plan.html924.html

Tobie introduced the work on his document, saying he'd fixed a range of issues and items that had been raised on various telecons.

A new section on conformance. http://coremob.github.com/coremob-2012/ED-coremob-20120924.html#conformance

Various points arose:

- a browser using a monochrome display is conformant to color related specifications, likewise conforms to touch related aspects when running on a non-touch enabled device. [with thanks (probably) to Dom: "technically, a device without a touch screen conforms to the Touch Event specs by virtue of the truth of predicates applied to the empty set"]

- given that the feature set is aspirational, and that some of the specifications won't be complete till after it has been completed, does it make sense to talk about conformance at all, at this stage? Should it be trimmed back to contain only "the usual suspects" in W3C terms, namely those specs that have reached an appropriate maturity level to form reliable dependencies?

- what about browsers running on devices/platforms that do not provide equal access - so one browser is conformant and another can't be because access to the relevant API / features is not public?

4. Discussion of Requirements and Review of Dependency of Coremob-2012 on them.

Noting that in a W3C context as contributed by Dom, conformance means "Fulfillment by a product, process, systems, or service of a specified set of requirements." [3] there is still a question of being clear what one means by conformance. Conformance to Requirements would mean that an appropriate set of Specifications exists to fulfil those requirements. Conformance to Specification would mean that an implementation claims to implement the features and implementation details of the Specification, and Conformance to Tests would mean that an implementation passes tests that claim to verify the correctness and completeness of the implementation.

[3] http://www.w3.org/TR/qaframe-spec/#glossary

There was a discussion about the nature of conformance to tests and that one needed to be careful since some conformance claims are not testable in some contexts, some cannot be automated and merely conforming to functional tests doesn't make a feature usable in which context Dom noted that tests assess whether something "could be conformant". Tobie pointed the group at the HTML(5) 2014 planned approach [4] to focus and test, describing it as "very sane" and also his document [5] (discussed later) on test approaches.

[4] http://dev.w3.org/html5/decision-policy/html5-2014-plan.html
[5] http://coremob.github.com/coremob-test-approach/

Jo asked Matt to introduce the work that he had collated [6] on the top apps in which people spend their time and what features are required for them to be built. Matt pointed out that there is a difference between people spending time and "top 10". 2D games may be popular but to where the most time is spent.

[6] https://docs.google.com/spreadsheet/ccc?key=0AuIhlK0fCwP4dEFPR1pUWHk1QVczcV9xbFAtX19CMXc

Earlier Tobie had noted that his announcement at the Palo Alto F2F that he'd started work on a Use Case document. Lars Bolstad of Opera and Giridhar Mandyam of Qualcomm said they were available to work on this document using the FB spreadsheet introduced by Matt Kelly as a starting point. Bryan Sullivan of AT&T  said that he could contribute. It was agreed that it would be a short document to capture Use Cases in categories  and then to note what features are required to fulfil those use cases. The time frame agreed was 2 weeks from the meeting for a first draft and a month for "last call". End of year for agreement. 

It was noted that the Requirements logically precede the Coremob 2012 document, since logically it would not make sense to specify technologies for implementation of features that had not been decided upon. 

Some members of the group expressed frustration that the expressed objective of publishing the document looked like it was in danger. Other members believed that publishing for the sake of publishing was not especially useful and that reducing the feature set in CoreMob 2012 because the specs are not ready would reduce its value and reduce the intended impact of communicating to W3C WGs that their work is urgently needed.

It was agreed that work on both should continue in parallel, and a resolution was taken  to publish CoreMob 2012 noting that there Use Cases and Requirements had not yet been published and noting that the document would likely change to some extent as a result of that. This resolution became moot as a result of later discussion.

5. Analysis of CoreMob 2012

We returned to discussion of CoreMob 2012 with Tobie leading.

We returned to a question that has been raised before of taking subsets of Specs. Noting that we'd decided before not to do so if at all possible. The spec in question being, in this case, HTML(5). Two issues - firstly that there are features we don't need (some discussion about identifying them) and secondly that things like Responsive Images are not in it. Also that AppCache is "horribly broken". We noted that these are features we need and really it doesn't matter where they are specified. 

Dom noted that he had a problem with matching his understanding of what we are trying to do with a "per spec approach. Dom's IRC comment is repeated here

<dom> I came up with a classifications per feature this morning, that I find hard to mould in this per-spec approach:
<dom> What features are needed to make the mobile Web successful
<dom> * features that are not defined in any spec
<dom> * features that are supposedly defined in any spec, but don't match our requirements
<dom> * features that are defined but not currently tested
<dom> * features that are ready

We decided that we want low latency audio. On Device Adaptation (things that appear in the viewport meta tag). Here we decided to slice the relevant specification (CSS-ADAPTATION). We deferred a question of whether we want the whole of SVG pending use case analysis.

We discussed at length the need to separate out concerns relating to the functionality of applications while running from those relating to the lifecycle. In the end we did not find consensus around wording that proposed: "This document will focus on features that need to be implemented by vendors to support intrinsic operation and functionality of apps rather than ancillary features that are necessary for lifecycle, monetisation, deployment or other aspects that are not core to the intrinsic operation of the application" 

We discussed but did not agree the inclusion of "Orientation Lock" because there is no standard spec that supports it. We did not agree its removal either.

It being the end of a long day the group agreed that it would not be possible to progress the discussion further and noted that the earlier resolution about publication of the document had become moot.

Day 2

6. Discussion of Paper on Test Frameworks

Tobie introduced his paper [7] discussing the requirements and the various options for achieving them, together with their pros and cons.

[7] http://coremob.github.com/coremob-test-approach/

To start with Tobie mentioned that there is terminological confusion, which he tried to reduce by including some definitions. Fantasai pointed out that Tobie's definition did not cover some aspects that are important (say) to CSS. Dom noted that the descriptions are specific to the Group. 

Fantasai:

1) testharness.js tests (automated JS) 
2) reftests (automated visual) 
3) self-describing tests (manual)

It was agreed that the framework should be suitable for running a variety of test cases - some fast and impactful, like Coremob, others slower but more detailed. It was also agreed that it wasn't practical to consider taking all the desirables into account and that we need something soon that can be developed later.

Jo summarised the discussion:

First we need a formal statement of requirements that are in-scope for the charter 
Second is architecture that matches the requirements, but will not be implemented in full yet due to resources 
Third is something that works, soon

Gavin Thomas from Vodafone took an action to start a draft of requirements of what our testing efforts should be (ACTION-56)

It was agreed that the focus would be on JS test, though it was acknowledged that this would not enable simple sharing of tests with CSS WG. Toby led the group through a discussion of current W3C test repository and the variety of approaches to committing tests from various WGs. It was noted that there were some dependencies on server side components and Tobie took an action to follow up on that (ACTION-57). A discussion of the JSON API revealed that it is at an early stage and Dom took and action (ACTION-58) to progress this. 

Noting that running all the tests associated with a particular spec might take longer than is desirable for a "fast imapctful" test, we took a resolution that the tests would be subsetted only to the extent that that made sense for the features we are interested in, and we'd consider the time aspect only after timeliness was shown to be a problem.

We took a resolution that we'd proceed with Tobie's Option 2 (described in the paper linked above []) though we noted that it didn't cover all the desirables and was based more on a pragmatic thought that we want something soon. We're not excluding the Options 3 or 4 for later development.

Noting that the next steps were to spec the test running and then implement it, Dom took ACTION-60 to start a wiki page outlining the tasks required to build the initial test framework (which is at [8]) and Bryan, Rob Shilston and Matt agreed to look at finding resources to follow on implementing. (ACTION-61, ACTION-62 and ACTION-63). Markus Leutwyler from HP said he'd help with the coding.

[8] http://www.w3.org/community/coremob/wiki/Todos#Defining_the_requirements_for_the_Test_Runner

Returning to the topic of what the group would be sourcing from Ringmark, it was stated that it was undecided as to whether current Ringmark tests at rng.io would be adopted or not. And it was further stated that this would depend on an analysis of what W3C tests exist, whether the WGs producing the specs CoreMob depends on are creating them, what gaps remain and whether the rng.io tests fill those needs.

It was noted that the group needed to produce a framework to run tests, a front end would be nice but that we'd previously discussed it being open to anyone to create a front end that runs the tests. Facebook might consider repurposing its Ringmark Front-End to depend on the framework and hence run the W3C authoritative tests.

7. Testing

Noting that Dom has a document [9] discussing "Standards for Web Applications on Mobile: Current State and Roadmap", having identified the gaps in testing we would need to look at the gaps that need plugging. Tobie reiterated his admiration for the HTML(5) Plan 2014 document. Tobie stated that he thought we shouldn't worry about things known good, but acknowledged that there is a problem of knowing that there is interoperability without there being tests to verify interoperability. We also discussed how individual contributors might create tests, contribute tests and submit them to a WG for adoption, or as an explanation of what they mean by something being broken. We noted that there are licensing issues relating to submitting tests to W3C.

[9] http://www.w3.org/2012/08/mobile-web-app-state/

We returned once again to a discussion of the test runner (framework) and the FB front end Coremob Front End - Tobie noted that he though "a reasonably good engineer" could produce his Option 2 in about a week and that the existing FB Coremob JS was about 2k lines of JS and could be "resued pretty effectively".

Re Gap Analysis between what exists in the form of W3C tests and what might be required for Coremob, Rob Shilston led an effort to do a preliminary pass [10] at the existing W3C tests. Various people volunteered to help do the assessment, by section of the current state of W3C specs. The process being:

"You open the sheet, for each line in the summary, assigned to you, you update your contact name to you,  you copy the template sheet, rename the sheet to match the spec and for each section of the spec  you assess test coverage  as a row in your sheet  and then you put a summary status on the summary sheet

[10] https://docs.google.com/spreadsheet/ccc?key=0At0Ot6R4q4ZadHZ0QWdVakpZWHE0QmdoM3BOMXdsSVE&pli=1#gid=0

We noted that there are other sources of tests that it would be useful to make reference to - noting again that there are licensing concerns plus they would probably not be compatible with our choice of testharness.js.

8. Review and Wrap

We decided not to progress discussions on Coremob 2012 at that point, since it had become clear that there were too many other items to proceed with in advance.

It was noted that some people were unhappy about the apparent lack of progress on Coremob 2012. It was noted that we had a path forward and that we had some route to getting it delivered based on a statement of requirements. Noting also that this document was a focus, for sure but is by no means the only deliverable. Jo noted that the group had been in existence for only 6 months and that it was inevitable that it would take time for the group to gain clarity and move forward. It should also be noted that as the group learns more the direction may continue to change.

Giri then presented on the Qualcomm Vellamo system level benchmarking initiative. Tobe discussed something he and Tomomi Imura Nokia had been thinking about - namely a Web app - a Camera app - as a demonstrator of problems performance issues etc.

We agreed that Wednesdays at 2pm London time was equally inconvenient to everyone as a time for future teleconferences (except those based in London). We tentatively agreed that based on projected progress we should meet F2F again, it was suggested that this should be alter January in Asia, and Jo took ACTION-65 to progress.

We thanked Mozilla for hosting us, we thanked Tobie for writing most of the documents and we thanked the scribes for scribing.

---
Jo Rabin
Chair