Re: [testing] initial results from testing priorities survey

WFM.

I'll use this criteria when generating the final result, unless I see a  
discussion on alternative approaches.

/g


On Sun, 23 Jun 2013 23:06:33 +0200, Vickers, Mark  
<Mark_Vickers@cable.comcast.com> wrote:

> Thanks! Works for me!
>
> mav
>
> On Jun 23, 2013, at 10:56 PM, "HU, BIN" <bh526r@att.com>
>  wrote:
>
>> Mark,
>>
>> Thank you for the great suggestion. So the mapping can be tweaked as:
>>
>> - Internal:
>>
>> OK: 4
>> (OK): 2
>> NO: 0
>>
>> - External:
>>
>> P/M/Now: 4
>> F/M/*:  3
>> P/O/Now: 2
>> F/O/*:  1
>> P/N:  0
>> F/N:  0
>> N/*:  0
>>
>> All,
>>
>> Feel free if you have other suggestions to the group.
>>
>> Thanks
>>
>> Bin
>>
>> -----Original Message-----
>> From: Vickers, Mark [mailto:Mark_Vickers@cable.comcast.com]
>> Sent: Sunday, June 23, 2013 9:32 AM
>> To: HU, BIN
>> Cc: Giuseppe Pascale; public-web-and-tv@w3.org
>> Subject: Re: [testing] initial results from testing priorities survey
>>
>> Bin,
>>
>> This is a great start. I only have a couple of suggested tweaks:
>>
>> First, I think I'd prioritize Mandatory for Present or Future over any  
>> Optional. Based on one reply I was involved with, the timing of future  
>> specs was as little as a few months from now, which is less time than  
>> these tests will take to create. The current Testing schedule is a two  
>> year program with a July start date. Whereas, the need for tests for  
>> optional parts of specs is by definition, well, optional. This also  
>> would mean that a 3 on the Internal survey would correspond to Optional  
>> on the External survey, which is how I think of it (i.e. "Nice to have,  
>> but  not required").
>>
>> Second, it might be better to reverse the numbering and use a 4 to 0  
>> rather than a 1 to 5 scale. That way, the lowest level rating of "No  
>> testing" is zero, which would be the same for any of the many specs  
>> that weren't included in the survey. No expressed need for testing is  
>> presented as 0.
>>
>> Thanks,
>> mav
>>
>>
>>
>> On Jun 21, 2013, at 11:08 PM, "HU, BIN" <bh526r@att.com> wrote:
>>
>>> Giuseppe, Clarke, and all,
>>>
>>> Thanks for collecting and putting those information together.
>>>
>>> The internal survey and external survey use different metrics, i.e.:
>>> - Internal: priorities
>>> - External: reference status / mandatory v.s. optional / timing
>>>
>>> If we want to aggregate those results into a single column, e.g.  
>>> priority level (1-5), I propose the mapping as follows:
>>>
>>> - Internal:
>>>
>>> OK: 1
>>> (OK): 3
>>> NO: 5
>>>
>>> - External:
>>>
>>> P/M/Now: 1
>>> P/O/Now: 2
>>> F/M/*:  3
>>> F/O/*:  4
>>> P/N:  5
>>> F/N:  5
>>> N/*:  5
>>>
>>> We can first map the individual results to the scale 1-5, and then  
>>> calculate the scores for each spec (i.e. the highest votes) to  
>>> conclude its final priority level.
>>>
>>> Feel free to suggest other methods for mapping the result and  
>>> calculating the score.
>>>
>>> Thanks
>>> Bin
>>>
>>> -----Original Message-----
>>> From: Giuseppe Pascale [mailto:giuseppep@opera.com]
>>> Sent: Friday, June 21, 2013 4:15 AM
>>> To: public-web-and-tv@w3.org
>>> Subject: [testing] initial results from testing priorities survey
>>>
>>> Clarke, all
>>> I've started to copy the results from the responses received on the  
>>> MEMBER
>>> ONLY wiki
>>>
>>> https://www.w3.org/Member/wiki/Webtv/testing
>>>
>>> FYI, I added the link above also to the testing section of the public  
>>> wiki
>>> http://www.w3.org/2011/webtv/wiki/Testing#External_Groups
>>>
>>> Also, I've closed the internal doodle poll, and the result are now
>>> available on the MEMBER ONLY wiki
>>> https://www.w3.org/Member/wiki/Webtv/testing#Internal_members_survey
>>>
>>> What remains to be done:
>>>
>>> 1. wait for a couple of groups that have promised a reply soon
>>> 2. copy those result on the MEMBER wiki, once received
>>> 3. aggregate the internal member results, add a column in the table for
>>> Web&TV
>>> 4. define how we want to aggregate all the result in order to provide a
>>> single "TV" column for the W3C testing group to consider. In particular
>>> decide which kind of metric to use, as we have, for each spec, a  
>>> variety
>>> of values we can consider:
>>>
>>> - how many group already reference it
>>> - how many group consider testing of it mandatory
>>> - how many groups plan to reference it in future
>>> - when testing would be needed
>>> - how it scored in the member survey
>>>
>>> What we could do is to define a priority level (e.g. 1-5). If we do  
>>> that,
>>> we need to define how we map the info above on a given priority level.
>>>
>>>
>>> /g
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> --
>>> Giuseppe Pascale
>>> Product Manager TV & Connected Devices
>>> Opera Software
>>>
>>


-- 
Giuseppe Pascale
Product Manager TV & Connected Devices
Opera Software

Received on Monday, 24 June 2013 10:02:24 UTC