Re: ARIA-AT CG Working Mode draft copy 2020-02-06

(This is for https://github.com/w3c/aria-at/issues/41)

New snapshot attached, this time in HTML since the document now has a table.

Den tors 6 feb. 2020 kl 15:36 skrev Simon Pieters <zcorpan@gmail.com>:

> (This is for https://github.com/w3c/aria-at/issues/41)
>
> # ARIA-AT CG Working Mode
>
> ## Tests
>
> ### Roles
>
> *   Researcher
> *   Test author
> *   Tester
> *   Implementer
> *   Web developer (or other interested party)
> *   Chair
>
> ### Research guide
>
> Conducting research for a test plan means to:
>
> *   Study existing web content in the wild, to look for how common
> different patterns are. This can be done through HTTP Archive, or with
> other means.
> *   Study the current behavior of browsers and screen readers. This can be
> done by creating a demo and testing it in different browser/screen reader
> combinations.
>
> (This guide needs to be expanded, and maybe moved elsewhere.)
>
> ### Process
>
> 1. (Test author) Document a use case.
> 2. (Test author) Write a hypothesis and draft test plan to be used as a
> basis for research.
> 3. (Researcher) Research existing web content.
> 4. (Researcher) Research existing implementations.
> 5. (Researcher) Report research findings.
> 6. (Test author) Propose a behavior that
>     *   meets the use case
>     *   is compatible with existing web content
>     *   implementers are willing to implement
> 7. (Test author) Write a test plan
> 8. (Implementer, Web developer) Review the proposed behavior and test plan
> (Timebox: X)
> 9. (Test author) Write the test
> 10. (Tester) Run the test
> 11. (Tester) Report results
> 12. (Implementer, Web developer) Review the test and test results
> (Timebox: X)
> 13. (Test author) File bugs for implementers
> 14. (Implementer) Fix the bug, or provide (late) feedback on the test
>
> Steps 2-5 are optional, but are generally encouraged. If a test has been
> removed, then new attempts to solve the same use case must have research.
>
> The chairs should seek wide review at a regular cadence.
>
> ### Conflict resolution
>
> If agreement cannot be made about what the tested behavior should be, it
> can be resolved as follows:
>
> *   If the feedback is late, then:
>     *   If the feedback does not bring any new information, the test
> author should respond and keep the test as-is.
>     *   If the feedback brings new information, then the test author
> should evaluate the feedback and make a decision on whether to redo steps
> 6-14, or respond and keep the test as-is.
> *   If the involved implementers agree with each other on what the
> behavior should be but disagree with the test author, then:
>     *   The test author should edit the test plan and the test to reflect
> implementer consensus, or escalate to the chairs.
>
> In any case, if agreement still cannot be made, either party can escalate
> to the chairs.
>
> ### Escalation to chairs
>
> (How should people escalate?)
>
> The chairs attempt to resolve the conflict. If this is not possible, the
> chairs can opt for one of the following:
>
> *   Find a solution that is mutually acceptable.
> *   Choose one behavior based on merit or based on majority implemented
> behavior.
> *   Remove the test. This should not be chosen if research was done for
> the test.
>
> The chairs should respond within X.
>
> ## Test Runner
>
> TODO
>
> ## GitHub management
>
> TODO
>
> --
> Simon Pieters
> https://bocoup.com/
>


-- 
Simon Pieters
https://bocoup.com/