ARIA-AT CG Working Mode draft copy 2020-02-06

(This is for https://github.com/w3c/aria-at/issues/41)

# ARIA-AT CG Working Mode

## Tests

### Roles

*   Researcher
*   Test author
*   Tester
*   Implementer
*   Web developer (or other interested party)
*   Chair

### Research guide

Conducting research for a test plan means to:

*   Study existing web content in the wild, to look for how common
different patterns are. This can be done through HTTP Archive, or with
other means.
*   Study the current behavior of browsers and screen readers. This can be
done by creating a demo and testing it in different browser/screen reader
combinations.

(This guide needs to be expanded, and maybe moved elsewhere.)

### Process

1. (Test author) Document a use case.
2. (Test author) Write a hypothesis and draft test plan to be used as a
basis for research.
3. (Researcher) Research existing web content.
4. (Researcher) Research existing implementations.
5. (Researcher) Report research findings.
6. (Test author) Propose a behavior that
    *   meets the use case
    *   is compatible with existing web content
    *   implementers are willing to implement
7. (Test author) Write a test plan
8. (Implementer, Web developer) Review the proposed behavior and test plan
(Timebox: X)
9. (Test author) Write the test
10. (Tester) Run the test
11. (Tester) Report results
12. (Implementer, Web developer) Review the test and test results (Timebox:
X)
13. (Test author) File bugs for implementers
14. (Implementer) Fix the bug, or provide (late) feedback on the test

Steps 2-5 are optional, but are generally encouraged. If a test has been
removed, then new attempts to solve the same use case must have research.

The chairs should seek wide review at a regular cadence.

### Conflict resolution

If agreement cannot be made about what the tested behavior should be, it
can be resolved as follows:

*   If the feedback is late, then:
    *   If the feedback does not bring any new information, the test author
should respond and keep the test as-is.
    *   If the feedback brings new information, then the test author should
evaluate the feedback and make a decision on whether to redo steps 6-14, or
respond and keep the test as-is.
*   If the involved implementers agree with each other on what the behavior
should be but disagree with the test author, then:
    *   The test author should edit the test plan and the test to reflect
implementer consensus, or escalate to the chairs.

In any case, if agreement still cannot be made, either party can escalate
to the chairs.

### Escalation to chairs

(How should people escalate?)

The chairs attempt to resolve the conflict. If this is not possible, the
chairs can opt for one of the following:

*   Find a solution that is mutually acceptable.
*   Choose one behavior based on merit or based on majority implemented
behavior.
*   Remove the test. This should not be chosen if research was done for the
test.

The chairs should respond within X.

## Test Runner

TODO

## GitHub management

TODO

-- 
Simon Pieters
https://bocoup.com/