Re: Next steps for Coremob-2012 and the group.

On 1/23/13 4:21 PM, "SULLIVAN, BRYAN L" <bs3131@att.com> wrote:

>I figured someone would have already done a similar analysis, but didn't
>know where to look. That's why I went ahead and put this on the wiki.
>
>I think we should develop automated updates to such an analysis, but
>publish a periodic (monthly or quarterly) report in the meantime. I am
>willing to help drive that effort.
>
>The "done-ness" of tests is a key question. As I did the analysis, I had
>to take various kinds of generalizing and summarizing actions that roll
>up the single numbers, and I know for sure that the value of the single
>numbers is low until we understand and validate what contributes to them.
>With so much variance in what's placed where and how it's identified in
>the lifecycle of tests, I found it very difficult to get more than a
>thumbnail sketch of the numbers. At least the ones with "0" were easier,
>if an unwelcome discovery!
>
>I think useful short-term actions are to:
>
>1) take a closer look at each suite, talk to the champion/contributors,
>and document their practice
>
>2) break out the test numbers in terms of the lifecycle stage e.g.
>a) submitted (note that it's unclear where tests of different authors may
>be duplicates)
>b) approved (meaning hopefully something consistent, e.g. reviewed and
>validated through execution with at least one UA, but optimally all major
>UAs, at least to validate that no test design errors are causing test
>failure with any UA)
>c)  active (meaning the tests have been incorporated into the test
>framework
>
>3) Add numbers indicating the nature of the tester experience:
>a) manual tests
>b) automated tests
>
>All of this is focused on aligning processes, identifying where resources
>are needed, and generally providing a more useful guide to what tests are
>available and in what form.
>
>Thanks,
>Bryan Sullivan

Thanks for your ongoing thought on this matter, Bryan.

Shared mine online recently[1]. This might be of interest to you.

Best,

--tobie

[1]: 
http://tobie.github.com/w3c-testing-plan/unofficial-w3c-testing-plan-201201
16.html

Received on Wednesday, 23 January 2013 15:53:19 UTC