RE: [testing] initial results from testing priorities survey

Mark,

Thank you for the great suggestion. So the mapping can be tweaked as:

- Internal:

OK:	4
(OK):	2
NO:	0

- External:

P/M/Now:	4
F/M/*:		3
P/O/Now:	2
F/O/*:		1
P/N:		0
F/N:		0
N/*:		0

All,

Feel free if you have other suggestions to the group.

Thanks

Bin

-----Original Message-----
From: Vickers, Mark [mailto:Mark_Vickers@cable.comcast.com] 
Sent: Sunday, June 23, 2013 9:32 AM
To: HU, BIN
Cc: Giuseppe Pascale; public-web-and-tv@w3.org
Subject: Re: [testing] initial results from testing priorities survey

Bin,

This is a great start. I only have a couple of suggested tweaks:

First, I think I'd prioritize Mandatory for Present or Future over any Optional. Based on one reply I was involved with, the timing of future specs was as little as a few months from now, which is less time than these tests will take to create. The current Testing schedule is a two year program with a July start date. Whereas, the need for tests for optional parts of specs is by definition, well, optional. This also would mean that a 3 on the Internal survey would correspond to Optional on the External survey, which is how I think of it (i.e. "Nice to have, but  not required").

Second, it might be better to reverse the numbering and use a 4 to 0 rather than a 1 to 5 scale. That way, the lowest level rating of "No testing" is zero, which would be the same for any of the many specs that weren't included in the survey. No expressed need for testing is presented as 0.

Thanks,
mav



On Jun 21, 2013, at 11:08 PM, "HU, BIN" <bh526r@att.com> wrote:

> Giuseppe, Clarke, and all,
> 
> Thanks for collecting and putting those information together.
> 
> The internal survey and external survey use different metrics, i.e.:
> - Internal: priorities
> - External: reference status / mandatory v.s. optional / timing
> 
> If we want to aggregate those results into a single column, e.g. priority level (1-5), I propose the mapping as follows:
> 
> - Internal:
> 
> OK:	1
> (OK):	3
> NO:	5
> 
> - External:
> 
> P/M/Now:	1
> P/O/Now:	2
> F/M/*:		3
> F/O/*:		4
> P/N:		5
> F/N:		5
> N/*:		5
> 
> We can first map the individual results to the scale 1-5, and then calculate the scores for each spec (i.e. the highest votes) to conclude its final priority level. 
> 
> Feel free to suggest other methods for mapping the result and calculating the score.
> 
> Thanks
> Bin
> 
> -----Original Message-----
> From: Giuseppe Pascale [mailto:giuseppep@opera.com] 
> Sent: Friday, June 21, 2013 4:15 AM
> To: public-web-and-tv@w3.org
> Subject: [testing] initial results from testing priorities survey
> 
> Clarke, all
> I've started to copy the results from the responses received on the MEMBER  
> ONLY wiki
> 
> https://www.w3.org/Member/wiki/Webtv/testing
> 
> FYI, I added the link above also to the testing section of the public wiki
> http://www.w3.org/2011/webtv/wiki/Testing#External_Groups
> 
> Also, I've closed the internal doodle poll, and the result are now  
> available on the MEMBER ONLY wiki
> https://www.w3.org/Member/wiki/Webtv/testing#Internal_members_survey
> 
> What remains to be done:
> 
> 1. wait for a couple of groups that have promised a reply soon
> 2. copy those result on the MEMBER wiki, once received
> 3. aggregate the internal member results, add a column in the table for  
> Web&TV
> 4. define how we want to aggregate all the result in order to provide a  
> single "TV" column for the W3C testing group to consider. In particular  
> decide which kind of metric to use, as we have, for each spec, a variety  
> of values we can consider:
> 
> - how many group already reference it
> - how many group consider testing of it mandatory
> - how many groups plan to reference it in future
> - when testing would be needed
> - how it scored in the member survey
> 
> What we could do is to define a priority level (e.g. 1-5). If we do that,  
> we need to define how we map the info above on a given priority level.
> 
> 
> /g
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> 
> -- 
> Giuseppe Pascale
> Product Manager TV & Connected Devices
> Opera Software
>

Received on Sunday, 23 June 2013 20:57:17 UTC