W3C home > Mailing lists > Public > public-web-and-tv@w3.org > June 2013

Re: [testing] initial results from testing priorities survey

From: Giuseppe Pascale <giuseppep@opera.com>
Date: Mon, 24 Jun 2013 07:36:49 +0200
To: ashimura@w3.org, Kazuyuki Ashimura <ashimura@w3.org>, "Vickers, Mark" <Mark_Vickers@cable.comcast.com>, "HU, BIN" <bh526r@att.com>
CC: "public-web-and-tv@w3.org" <public-web-and-tv@w3.org>
Message-ID: <a1adbf58-722c-443b-956a-58eb9d99c4e4@email.android.com>
Please remember that the non aggregated results are member confidential, so not ro be discussed on this list. 

Also remember that we are still waiting for a response or two.
-- 
Sent from my phone

Kazuyuki Ashimura <ashimura@w3.org> wrote:

>Thanks a lot for your initial discussion, Bin and Mark.
>
>It is fine by me to merge the internal/external results
>based on some measure (e.g., the following).  However,
>maybe we should be able to analyze/discuss the results
>separately based on the results themselves a bit more
>before merging.
>
>What do you think?
>
>Kazuyuki
>
>
>On 06/24/2013 06:06 AM, Vickers, Mark wrote:
>> Thanks! Works for me!
>>
>> mav
>>
>> On Jun 23, 2013, at 10:56 PM, "HU, BIN" <bh526r@att.com>
>>   wrote:
>>
>>> Mark,
>>>
>>> Thank you for the great suggestion. So the mapping can be tweaked
>as:
>>>
>>> - Internal:
>>>
>>> OK:	4
>>> (OK):	2
>>> NO:	0
>>>
>>> - External:
>>>
>>> P/M/Now:	4
>>> F/M/*:		3
>>> P/O/Now:	2
>>> F/O/*:		1
>>> P/N:		0
>>> F/N:		0
>>> N/*:		0
>>>
>>> All,
>>>
>>> Feel free if you have other suggestions to the group.
>>>
>>> Thanks
>>>
>>> Bin
>>>
>>> -----Original Message-----
>>> From: Vickers, Mark [mailto:Mark_Vickers@cable.comcast.com]
>>> Sent: Sunday, June 23, 2013 9:32 AM
>>> To: HU, BIN
>>> Cc: Giuseppe Pascale; public-web-and-tv@w3.org
>>> Subject: Re: [testing] initial results from testing priorities
>survey
>>>
>>> Bin,
>>>
>>> This is a great start. I only have a couple of suggested tweaks:
>>>
>>> First, I think I'd prioritize Mandatory for Present or Future over
>any Optional. Based on one reply I was involved with, the timing of
>future specs was as little as a few months from now, which is less time
>than these tests will take to create. The current Testing schedule is a
>two year program with a July start date. Whereas, the need for tests
>for optional parts of specs is by definition, well, optional. This also
>would mean that a 3 on the Internal survey would correspond to Optional
>on the External survey, which is how I think of it (i.e. "Nice to have,
>but  not required").
>>>
>>> Second, it might be better to reverse the numbering and use a 4 to 0
>rather than a 1 to 5 scale. That way, the lowest level rating of "No
>testing" is zero, which would be the same for any of the many specs
>that weren't included in the survey. No expressed need for testing is
>presented as 0.
>>>
>>> Thanks,
>>> mav
>>>
>>>
>>>
>>> On Jun 21, 2013, at 11:08 PM, "HU, BIN" <bh526r@att.com> wrote:
>>>
>>>> Giuseppe, Clarke, and all,
>>>>
>>>> Thanks for collecting and putting those information together.
>>>>
>>>> The internal survey and external survey use different metrics,
>i.e.:
>>>> - Internal: priorities
>>>> - External: reference status / mandatory v.s. optional / timing
>>>>
>>>> If we want to aggregate those results into a single column, e.g.
>priority level (1-5), I propose the mapping as follows:
>>>>
>>>> - Internal:
>>>>
>>>> OK:	1
>>>> (OK):	3
>>>> NO:	5
>>>>
>>>> - External:
>>>>
>>>> P/M/Now:	1
>>>> P/O/Now:	2
>>>> F/M/*:		3
>>>> F/O/*:		4
>>>> P/N:		5
>>>> F/N:		5
>>>> N/*:		5
>>>>
>>>> We can first map the individual results to the scale 1-5, and then
>calculate the scores for each spec (i.e. the highest votes) to conclude
>its final priority level.
>>>>
>>>> Feel free to suggest other methods for mapping the result and
>calculating the score.
>>>>
>>>> Thanks
>>>> Bin
>>>>
>>>> -----Original Message-----
>>>> From: Giuseppe Pascale [mailto:giuseppep@opera.com]
>>>> Sent: Friday, June 21, 2013 4:15 AM
>>>> To: public-web-and-tv@w3.org
>>>> Subject: [testing] initial results from testing priorities survey
>>>>
>>>> Clarke, all
>>>> I've started to copy the results from the responses received on the
>MEMBER
>>>> ONLY wiki
>>>>
>>>> https://www.w3.org/Member/wiki/Webtv/testing
>>>>
>>>> FYI, I added the link above also to the testing section of the
>public wiki
>>>> http://www.w3.org/2011/webtv/wiki/Testing#External_Groups
>>>>
>>>> Also, I've closed the internal doodle poll, and the result are now
>>>> available on the MEMBER ONLY wiki
>>>>
>https://www.w3.org/Member/wiki/Webtv/testing#Internal_members_survey
>>>>
>>>> What remains to be done:
>>>>
>>>> 1. wait for a couple of groups that have promised a reply soon
>>>> 2. copy those result on the MEMBER wiki, once received
>>>> 3. aggregate the internal member results, add a column in the table
>for
>>>> Web&TV
>>>> 4. define how we want to aggregate all the result in order to
>provide a
>>>> single "TV" column for the W3C testing group to consider. In
>particular
>>>> decide which kind of metric to use, as we have, for each spec, a
>variety
>>>> of values we can consider:
>>>>
>>>> - how many group already reference it
>>>> - how many group consider testing of it mandatory
>>>> - how many groups plan to reference it in future
>>>> - when testing would be needed
>>>> - how it scored in the member survey
>>>>
>>>> What we could do is to define a priority level (e.g. 1-5). If we do
>that,
>>>> we need to define how we map the info above on a given priority
>level.
>>>>
>>>>
>>>> /g
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>>>> Giuseppe Pascale
>>>> Product Manager TV & Connected Devices
>>>> Opera Software
>>>>
>>>
>>
>>
>
>
>-- 
>Kaz Ashimura, W3C Staff Contact for Web&TV, MMI and Voice
>Tel: +81 466 49 1170
Received on Monday, 24 June 2013 05:37:31 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:57:16 UTC