W3C home > Mailing lists > Public > public-html-testsuite@w3.org > February 2013

Re: Marking non-automated tests

From: Wolfram Kriesing <wk@uxebu.com>
Date: Thu, 7 Feb 2013 09:58:18 +0100
Message-ID: <CACQ4vkHCiwb8jCZkVuH0eCPHpA+_CUe=bwBy6GoTbBZ8z9op+g@mail.gmail.com>
To: Robin Berjon <robin@w3.org>
Cc: "public-html-testsuite@w3.org" <public-html-testsuite@w3.org>
I am following the list quietly, with interest :).
Maybe I introduce myself a bit. I am Wolfram Kriesing, from uxebu,
a small JavaScript company. I had been working on an alike project
a web test suite, that was open sourced by vodafone
https://github.com/vf/web-testsuite
I didn't work on it lately, but I see huge value in this kinda thing.

When I saw the topic of manual tests appearing here, my trigger
was pulled :). I see manual tests is one of the important and inevitable things
a test suite includes. Even running the danger that it is a little off topic:
I am curious if there are any discussions or plans to
1) host the test for execution and (the imho important thing)
2) to collect all the data
What do i mean? maybe it becomes obvious when running one of the
simple tests we used to build in that old test suite
http://static.uxebu.com/~cain/web-testsuite/src/tests-w3c-mediaquery-orientation.html
the test ist manual and the results are sent back to a server, so
we can generate stats about pass/fail on certain browsers
and devices ... unfortunately we never generated any stats yet :(

If this is a topic I would be interested in helping on it, I see a lot
of potential and things to do there :) Can I read up on the current plans
regarding this somewhere?

--
Kind regards / Saludos / Mit freundlichen Grüßen

Wolfram Kriesing - CTO, Co-Founder

Flash on the iPad? We make it work!
kriesing@uxebu.com, mobile: +49 174 300 4595
uxebu Inc.


On Wed, Feb 6, 2013 at 2:24 PM, Robin Berjon <robin@w3.org> wrote:
> Hi gang,
>
> I've been running some analyses on the tests we have in the suite, and I'm
> noticing a rather large bunch of tests that aren't properly automated. I
> knew there were some, but I hadn't realised it was this many.
>
> A lot of those I've seen can be converted to testharness, and should be.
> I'll be producing a list of all conversion candidates.
>
> But some tests just have to remain as manual or reftests. We've had some
> conventions to mark those up, but nothing seems to have been used
> consistently at this point.
>
> The existing conventions seem to be mostly about using <meta>. I was
> wondering if we shouldn't just use something simpler. How about
>
>   <html data-manual>
>
> ?
>
> --
> Robin Berjon - http://berjon.com/ - @robinberjon
>
Received on Thursday, 7 February 2013 11:04:27 GMT

This archive was generated by hypermail 2.2.0+W3C-0.50 : Thursday, 7 February 2013 11:04:27 GMT