Re: Browser vendor meeting on testing, October 28th

Phillipe,

I do intend to be there to hear specifically the browser vendor input as you
note, and I can also describe briefly the test framework we have developed
in WAC and its basis in / dependency upon test frameworks and assets,
including those developed by W3C. We have already certified several devices
and widget runtime (WRT) clients under WAC (
http://www.wacapps.net/devices-and-runtimes), based upon the WAC APIs and
W3C standards profile that we defined there (
http://specs.wacapps.net/2.0/jun2011/). The WAC test system is at
http://tests.wacapps.net/2.0/, and you can see there that there is a strong
reuse of W3C test assets, and a lot of focus has been put into automation
where possible.

But my main intent in participating would be to learn what challenges
browser vendors see in continual testing for Web standards compliance. Since
many are moving to a rapid/continual release cycle, the potential for
regression is significant and thus there is significant need for
efficiency/automation and optimized test coverage.

The release cycle for mobile implementations in comparison is still at most
two-three times per year, and "downstream" products that are based upon them
(e.g. the WAC WRTs) have thus a different type of need, that of baselining
more fully the compliance of the underlying Web runtime platform version,
and providing a comprehensive test suite that covers the numerous
device-specific adaptations/extensions that inevitably result. It's my goal
for the Web Testing Interest Group that the W3C Web testing framework and
assets also support that type of production cycle.

Thanks,
Bryan Sullivan

On Fri, Oct 7, 2011 at 2:53 PM, Philippe Le Hegaret <plh@w3.org> wrote:

> Thank you Wilhelm for sending this early announcement.
>
> An additional note:
>
> The goal of the meeting is to help that W3C Working Groups release their
> tests in a sensible format and also can make use of tests produced by
> others.
>
> Our primary emphasis is to leverage the experience of browser vendors in
> test format and test framework. However, if you're not a browser vendor
> and you think you have an interesting story to tell, feel free to come
> and tell us. We'd love to learn how you are testing the Open Web
> Platform! Depending on your story, we may ask you to present it as well.
> The room space is however limited and hopefully we won't have to make
> hard choices. This meeting is not meant as a full kickoff meeting for
> the Web Testing IG but we're trying to take advantage of being there to
> exchange ideas.
>
> Philippe
>
> On Fri, 2011-09-23 at 17:58 +0200, Wilhelm Joys Andersen wrote:
> > Hi,
> >
> > On October 28th, there will be a meeting between representatives from
> > different browser vendors in Mountain View, California. We will discuss
> > how to best test web technologies in browsers and how to make browser
> > vendors actively use and contribute to W3C test suites.
> >
> > The full invitation sent to Apple, Google, Microsoft, Mozilla and Opera
> > is included below. (If I have forgot anyone, please let me know! :)
> >
> > ----------------------
> >
> > The Web Testing Interest Group at W3C[1], of which I am co-chair,
> > would like to invite you (or someone from your team) to a meeting on
> > testing of web browsers in Mountain View on Friday the 28th of October
> > 2011. (TPAC starts in the same neighbourhood the following Monday.)
> >
> > The goal of the meeting is to reach a rough consensus on what testing
> > methods and frameworks W3C working groups should use. Naturally, we
> > would like representatives of as many browser vendors as possible to
> > attend, as we wish to:
> >
> >    * ensure that tests developed by W3C working groups
> >      are usable by and useful for browser vendors
> >
> >    * make it as easy as possible for browser vendors to
> >      contribute tests to W3C working groups
> >
> > The primary test types we wish to discuss inititially are:
> >
> >    * Tests checking whether something is rendered correctly
> >      (visual tests, reference tests).
> >
> >    * Tests that can be evaluated through JavaScript.
> >
> >    * Tests that require some sort of user interaction
> >      (typically automated with Selenium/WebDriver/Watir).
> >
> > Each browser vendor is invited to give a short presentation (up to 25
> > minutes) on how they currently test their browser, and how tests from
> > W3C working groups fit into the rest of the testing they do.
> > (Presenting is encouraged, but not required.)
> >
> > In addition, there will be presentations of different frameworks and
> > methods for cross-browser testing of web standards support. Some widely
> > deployed, others merely proposed.
> >
> > To sign up, please fill in your name here:
> >
> >    http://www.w3.org/2002/09/wbs/99999/browser-testing-meeting/
> >
> > If you wish to bring along one or two others from your team, you're
> > welcome to do so. They, too, must add their names to the list of
> > attendees above.
> >
> > If you have any questions, feel free to contact me via email, IRC
> > (wilhelm on w3.org and freenode) or telephone (+47 97723641).
> >
> >
> > Practical information:
> >
> >    Venue: Google headquarters, Mountain View, building 1950,
> >           meeting room 1950-1-Arroyo Valle.
> >
> >    Time:  Friday 28th of October, 09:30 - 16:00
> >
> > [1]: http://www.w3.org/2011/05/testing-ig-charter.html
> >
>
>
>
>
>