Minutes of the joint AGWG-Silver virtual meeting of 29 April 2021

== Summary ==

 1. Testing objectivity
    <https://www.w3.org/2021/04/29-ag-minutes.html#t01>  We agreed that
    instead of framing the conversation as subjective/objective, that we
    would pursue improving reliability between different testers.
 2. Which tests to include in conformance
    <https://www.w3.org/2021/04/29-ag-minutes.html#t02>  We agreed that
    AGWG members would volunteer to work through the tests in WCAG3 Text
    Alternatives guideline to pursue the idea of increased granularity
    and proof of concept of the viability of adjectival ratings. 
    Contact Jeanne or Makoto if you are interested in helping.  We also
    discussed where Assistive Technology testing fit, but did not reach
    a conclusion.
 3. Scoring Issues <https://www.w3.org/2021/04/29-ag-minutes.html#t03> 
    Wide-ranging discussion of processes and views, critical errors, the
    place for A, AA, AAA guidance, disability equity, gaming the system,
    scoping, automated testing, what is in a process, if and how heavy a
    burden testing is in the proposal in the FPWD,  VPATs and reporting,
    creating building blocks to support different  reporting and
    tracking systems, how to determine the impact of barriers, and the
    realization that a binary scoring system is just a rating system
    with only two options.  We agreed to create tests and then try
    scoring them with different options: 2, 3 and 5 (and maybe more).
 4. Session 3 - Conformance
    <https://www.w3.org/2021/04/29-ag-minutes.html#t04>
     1. Discussion of whether there should be a level lower than Bronze
        and should it be based on automation.  We resolved to support a
        simpler on-ramp below bronze level that is not conforming and
        agreed we will explore this further later.
     2. We also discussed whether the Bronze Silver Gold levels should
        be based on a subset of tests (similar to WCAG2) or a threshold
        of test results (if you score [X] you get Bronze). The group
        generally agreed that they preferred a threshold, but the chairs
        took an action to clarify the question further with examples.
     3. We discussed whether to consider including a Maturity Model
        document in WCAG3. There was a high level discussion of what
        could be in such a document and where it could be positioned.
        The sense of the group was that it should be pursued as a
        separate document. Some argued that it could be a separate
        document that is referenced by WCAG3, others did not want to
        commit to any publishing.  We agreed to continue to discuss it
        in the future.  It was late and there was a rush to finish, but
        there were objections raised after the resolution went in the
        minutes.

== Resolutions ==

 1. For WCAG 3, testing will aim to improve inter-tester reliability and
    will work on testing to measure this
    <https://www.w3.org/2021/04/29-ag-minutes.html#r02>
 2. We support a simpler on-ramp below bronze that is not conformance,
    and will explore this further later
    <https://www.w3.org/2021/04/29-ag-minutes.html#r03>
 3. We will pursue publishing Maturity Model as a separate document.
    <https://www.w3.org/2021/04/29-ag-minutes.html#r04>

== Action Items ==

Action: Rachael and chairs to draw up examples of setting levels by 
threshold or test subsets 
<https://www.w3.org/2021/04/29-ag-minutes.html#a01>

== Minutes ==

https://www.w3.org/2021/04/29-ag-minutes.html

== Questions ==

This is a new feature of IRC that allows us to store questions that come 
up that are tangential to the discussion. We did try to address them 
during Session 3, but they are not captured in the Minutes and it seemed 
useful to save them.

  Zakim Q1: Should we have a structure where a site or product must meet 
all automated tests before doing manual or qualitative tests?
* Zakim Q2: to look at the inverted score where the barriers or errors 
are counted instead of the overall score.
* Zakim Q3: How do we address attempted gaming?
* Zakim Q4: Should critical errors be across a process or by the view?
* Zakim Q5: How do we score content-free components or templates?
* Zakim Q6: Discuss more about how to handle frameworks and CMSes