Re: [Minutes] WCAG Accessibility Conformance Testing (ACT) Task Force - Meeting on 12 April 2018

Dear WCAG ACT TF,

Level Access are planning to publish a good number of our Test Cases in the near future – as we believe this openness will be very useful for us, our Clients and the broader community.

To move things along we’re trying to follow the current template, but we're experiencing niggles in terms of its real-world implementation:

1) Our test cases are predominantly distinct snippets of testable content (HTML, etc…) which cause a single test to either be determined as “passed | failed | non-applicable”.  So, by having distinct “passed:”, “failed:” attributes we find that we have to include a good deal of empty attribute values e.g. "passed:;"... 

2) As said, we really also want to be able to provide examples of code for which a test will provide a "non-applicable" outcome; as we believe this is also very important.  However, no "non-applicable" outcome is available in the model.

3) Finally, we want to be able to say what system / browser the tests were run successfully on - as Chrome / Firefox / IE / etc... have different versions, are potentially different on different systems, and have different mechanisms for representing some elements.
    
Correcting the speed-bumps that we're experiencing would enable us to move more quickly to our (the Group's) desired outcome.

I wonder, if others are experiencing similar issues with the current model might we be open to considering some minor "non-breaking" changes.   

For example: 

- Could we as a group consider the inclusion of a "non-applicable" field.  Even though to me it is a super-ugly model, and I know there are better ways to doing this, my suggestion is made to ensure it is non-breaking to others.  
- Also, might the group be open to the inclusion of "test environment" as a field in the model - again, as a non-breaking change.

All the best 

Alistair
 
---
 
Alistair Garrison
Director of Accessibility Research
Level Access
 
On 16/04/2018, 22:10, "Shadi Abou-Zahra" <shadi@w3.org> wrote:

    [removing AGWG list from the thread]
    
    Thank you for re-raising this Alistair. I think last week we concluded 
    that we need to continue this discussion. Here is the prior proposal:
      - 
    https://www.w3.org/WAI/GL/task-forces/conformance-testing/wiki/Testing_Resources#Test_case_format_.28explicit_results.29

    
    Here is also the prior ACT TF discussion on this topic:
      - https://lists.w3.org/Archives/Public/public-wcag-act/2018Jan/0007

    
    Best,
       Shadi
    
    
    On 16/04/2018 20:54, Alistair Garrison wrote:
    > Hi,
    > 
    > Shame I missed the meeting, I was on holiday.
    > 
    > At the ICT Symposium last year a number of us were on a panel together and talked about opening up our test cases to the public.
    > 
    > During the debate, people mentioned that it was too difficult to have a single repository which centrally holds all the test cases; or that needs someone to accept them.
    > 
    > Instead, it might be best to look to create a simple "lightweight" interface format - which simply says when I publish a test case publically on my own site I will make certain that certain information is included with my test case.
    > 
    > This way, we can all write interpreters which enable us to run each others test cases - as we know they adhere to an interface.  I'm using interface in the software specific way - e.g. to be a test case, certain information must be available through known attribute names.
    > 
    > It's all rather simple really - we'd just have to agree some attributes.
    > 
    > The benefits are gigantic...  not least that people will be able to compare tools; but also that false positives in test cases will be identifiable - e.g. if 3 tools disagree on an outcome something must be wrong.
    > 
    > If nothing is to be done through ACT, I think I'll push to set up a short-lifespan interest group for this specific purpose.
    > 
    > Alistair
    > ---
    > Alistair Garrison
    > Director of Accessibility Research
    > Level Access
    > 
    > On 12/04/2018, 17:50, "Shadi Abou-Zahra" <shadi@w3.org> wrote:
    > 
    >      Hello,
    >      
    >      Minutes of the WCAG Accessibility Conformance Testing (ACT) Task Force
    >      teleconference meeting on Thursday 12 April 2018:
    >        - https://www.w3.org/2018/04/12-wcag-act-minutes

    >      
    >      Next meeting: *Thursday 19 April 2018*
    >      
    >      Regards,
    >         Shadi
    >      
    >      --
    >      Shadi Abou-Zahra - http://www.w3.org/People/shadi/

    >      W3C Web Accessibility Initiative (WAI)
    >      
    >      
    > 
    
    -- 
    Shadi Abou-Zahra - http://www.w3.org/People/shadi/

    Accessibility Strategy and Technology Specialist
    Web Accessibility Initiative (WAI)
    World Wide Web Consortium (W3C)

Received on Thursday, 19 April 2018 10:50:40 UTC