Re: [testing] initial results from testing priorities survey

On Mon, Jun 24, 2013 at 2:36 PM, Giuseppe Pascale <giuseppep@opera.com>wrote:

> Please remember that the non aggregated results are member confidential,
> so not ro be discussed on this list.
>
> Also remember that we are still waiting for a response or two.
>

Thank you for your clarification, Giuseppe.

So we should hold the detailed discussion on the Member
list first, shouldn't we?

Thanks,

Kazuyuki


-- 
> Sent from my phone
>
> Kazuyuki Ashimura <ashimura@w3.org> wrote:
>>
>> Thanks a lot for your initial discussion, Bin and Mark.
>>
>> It is fine by me to merge the internal/external results
>> based on some measure (e.g., the following).  However,
>> maybe we should be able to analyze/discuss the results
>> separately based on the results themselves a bit more
>> before merging.
>>
>> What do you think?
>>
>> Kazuyuki
>>
>>
>> On 06/24/2013 06:06 AM, Vickers, Mark wrote:
>>
>>> Thanks! Works for me!
>>>
>>> mav
>>>
>>> On Jun 23, 2013, at 10:56 PM, "HU, BIN" <bh526r@att.com>
>>> wrote:
>>>
>>> Mark,
>>>>
>>>> Thank you for the great suggestion. So the mapping can be tweak!
>>>>  ed
>>>> as:
>>>>
>>>>
>>>> - Internal:
>>>>
>>>> OK: 4
>>>> (OK): 2
>>>> NO: 0
>>>>
>>>> - External:
>>>>
>>>> P/M/Now: 4
>>>> F/M/*:  3
>>>> P/O/Now: 2
>>>> F/O/*:  1
>>>> P/N:  0
>>>> F/N:  0
>>>> N/*:  0
>>>>
>>>> All,
>>>>
>>>> Feel free if you have other suggestions to the group.
>>>>
>>>> Thanks
>>>>
>>>> Bin
>>>>
>>>> -----Original Message-----
>>>> From: Vickers, Mark [mailto:Mark_Vickers@cable.comcast.com]
>>>> Sent: Sunday, June 23, 2013 9:32 AM
>>>> To: HU, BIN
>>>> Cc: Giuseppe Pascale; public-web-and-tv@w3.org
>>>> Subject: Re: [testing] initial results from testing priorities survey
>>>>
>>>> Bin,
>>>>
>>>> This is a great start. I only have a couple of suggested tweaks:
>>>>
>>>> First, I think I'd prioritize Mandatory for Present or Future over any Optional. Based on one reply I was involved with, the timing of future specs was as little as a few months from now, which is less time than these tests will take to create. The current Testing schedule is a two year
>>>> program with a July start date. Whereas, the need for tests for optional parts of specs is by definition, well, optional. This also would mean that a 3 on the Internal survey would correspond to Optional on the External survey, which is how I think of it (i.e. "Nice to have, but  not required").
>>>>
>>>> Second, it might be better to reverse the numbering and use a 4 to 0 rather than a 1 to 5 scale. That way, the lowest level rating of "No testing" is zero, which would be the same for any of the many specs that weren't included in the survey. No expressed need for testing is presented as 0.
>>>>
>>>> Thanks,
>>>> mav
>>>>
>>>>
>>>>
>>>> On Jun 21, 2013, at 11:08 PM, "HU, BIN" <bh526r@att.com> wrote:
>>>>
>>>> Giuseppe, Clarke, and all,
>>>>>
>>>>> Thanks for collecting and putting those information together.
>>>>>
>>>>> The internal survey and exte!
>>>>>  rnal
>>>>> survey use different metrics, i.e.:
>>>>>
>>>>> - Internal: priorities
>>>>> - External: reference status / mandatory v.s. optional / timing
>>>>>
>>>>> If we want to aggregate those results into a single column, e.g. priority level (1-5), I propose the mapping as follows:
>>>>>
>>>>> - Internal:
>>>>>
>>>>> OK: 1
>>>>> (OK): 3
>>>>> NO: 5
>>>>>
>>>>> - External:
>>>>>
>>>>> P/M/Now: 1
>>>>> P/O/Now: 2
>>>>> F/M/*:  3
>>>>> F/O/*:  4
>>>>> P/N:  5
>>>>> F/N:  5
>>>>> N/*:  5
>>>>>
>>>>> We can first map the individual results to the scale 1-5, and then calculate the scores for each spec (i.e. the highest votes) to conclude its final priority level.
>>>>>
>>>>> Feel free to suggest other methods for mapping the result and calculating the score.
>>>>>
>>>>> Thanks
>>>>> Bin
>>>>>
>>>>> -----Original Message-----
>>>>> From: Giuseppe Pascale [mailto:giuseppep@opera.com]
>>>>> Sent: Friday, June 21, 2013 4:15 AM
>>>>> To: public-web-and-tv@w3.org
>>>>> Subject: [testing] initial results from testing prioriti!
>>>>>  es
>>>>> survey
>>>>>
>>>>> Clarke, all
>>>>> I've started to copy the results from the responses received on the MEMBER
>>>>> ONLY wiki
>>>>>
>>>>> https://www.w3.org/Member/wiki/Webtv/testing
>>>>>
>>>>> FYI, I added the link above also to the testing section of the public wiki
>>>>> http://www.w3.org/2011/webtv/wiki/Testing#External_Groups
>>>>>
>>>>> Also, I've closed the internal doodle poll, and the result are now
>>>>> available on the MEMBER ONLY wiki
>>>>> https://www.w3.org/Member/wiki/Webtv/testing#Internal_members_survey
>>>>>
>>>>> What remains to be done:
>>>>>
>>>>> 1. wait for a couple of groups that have promised a reply soon
>>>>> 2. copy those result on the MEMBER wiki, once received
>>>>> 3. aggregate the internal member results, add a column in the table for
>>>>> Web&TV
>>>>> 4. define how we want to aggregate all the result in order to provide a
>>>>> single "TV" column for the W3C testing group to consider. In particular
>>>>> decide which kind of metric to use, as we have, for each spec, a variety
>>>>> of values we can consider:
>>>>>
>>>>> - how many group already reference it
>>>>> - how many group consider testing of it mandatory
>>>>> - how many groups plan to reference it in future
>>>>> - when testing would be needed
>>>>> - how it scored in the member survey
>>>>>
>>>>> What we could do is to define a priority level (e.g. 1-5). If we do that,
>>>>> we need to define how we map the info above on a given priority level.
>>>>>
>>>>>
>>>>> /g
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>> --
>>>>> Giuseppe Pascale
>>>>> Product Manager TV & Connected Devices
>>>>> Opera Software
>>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>>
>>>> --
>> Kaz Ashimura, W3C Staff Contact for Web&TV, !
>>  MMI and
>> Voice
>>
>> Tel: +81 466 49 1170
>>
>>


-- 
Kaz Ashimura, W3C Staff Contact for Web&TV, MMI and Voice
Tel: +81 466 49 1170