Re: WCAG-EM Report Tool

Hi Richard, all,

Also note that this is a report generator tool based on WCAG-EM, not a 
WCAG-EM report validator. Of course we want to help people to create 
valid WCAG-EM reports but the tool has broader uses. Also, as Wilco 
pointed out, there may be times when the report is temporarily not 
meeting the WCAG-EM requirements. We have the "indicator" added to the 
issues list (issue #73):
  - https://github.com/w3c/wcag-em-report-tool/issues

Best,
   Shadi


On 13.8.2014 01:19, Wilco Fiers wrote:
> Hi Richard,
> Yes that is actually what it would be used for, to indicator, to indicate that the results aren't complete yet. Everything starts as untested, and when you're done everything is set to passed/failed/not present. You are right that a completed audit wouldn't have anything set to not checked.
>
> Wilco
>
>
> -------- Oorspronkelijk bericht --------
> Van: Richard <richard.warren@userite.com>
> Datum:12-08-2014 18:39 (GMT+01:00)
> Aan: ryladog@gmail.com, emmanuelle@sidar.org, Wilco Fiers <w.fiers@accessibility.nl>, gavin.evans@digitalaccessibilitycentre.org, v.conway@webkeyit.com
> Cc: shadi@w3.org, public-wai-evaltf@w3.org
> Onderwerp: RE: WCAG-EM Report Tool
>
> Sorry to be a pain but we cannot use terms like "cannot tell" or "not tested". These terms indicate that the site was not proprly checkrd and therefore makes the result meaningless.
>
> The only alternative to pass/fail is "not present " (which is. in effect. a pass as there is no barrier created)
>
> Richard (by the pool)
>
> Sent from Samsung tablet
>
>
>
> Katie Haritos-Shea GMAIL <ryladog@gmail.com> wrote:
>
>
> +1
>
>
>
> * katie *
>
> Katie Haritos-Shea
> Senior Accessibility SME (WCAG/Section 508/ADA/AODA)
>
> Cell: 703-371-5545 | ryladog@gmail.com | Oakton, VA | LinkedIn Profile |
> Office: 703-371-5545
>
> -----Original Message-----
> From: Emmanuelle Gutiérrez y Restrepo [mailto:emmanuelle@sidar.org]
> Sent: Tuesday, August 12, 2014 7:04 AM
> To: 'Wilco Fiers'; 'Gavin Evans'; 'WebKeyIT'
> Cc: 'Shadi Abou-Zahra'; 'Eval TF'
> Subject: RE: WCAG-EM Report Tool
>
> I agree with Wilco.
>
> "Cannot tell" is very useful.
>
> Best,
>
> Emmanuelle Gutiérrez y Restrepo
> Patrono y Directora General
> Fundación Sidar - Acceso Universal
> Email: coordina@sidar.org
> Personal: Emmanuelle@sidar.org
> Web: http://sidar.org
>
>
> -----Mensaje original-----
> De: Wilco Fiers [mailto:w.fiers@accessibility.nl] Enviado el: martes, 12 de
> agosto de 2014 10:23
> Para: Gavin Evans; WebKeyIT
> CC: Shadi Abou-Zahra; Eval TF
> Asunto: RE: WCAG-EM Report Tool
>
> Hi everyone,
>
> Interesting discussion! I've also had the following issue suggested on
> Github:
> https://github.com/w3c/wcag-em-report-tool/issues/69
>
> Renaming Untested to Not checked and Inapplicable to Not present seems like
> a good idea to me. I don't think Cannot tell should be removed though. Of
> course in the final report, if there is some criterion set to Cannot tell
> then the website wasn't fully audited, which should be clearly indicated.
> But there are quite a few scenarios in which a criterion can be set to
> Cannot tell. The main one I think would be if initial data was generated by
> an external source. Tools that test part of a criterion shouldn't report
> that criterion as passed or failed, but as failed or cannot tell. And
> because Cannot tell is part of EARL it's quite plausible that tools will
> provide this result.
>
> If nothing else, at least it should be shown if this is the case. Another
> use case would be when multiple evaluators work on the same audit. A junior
> auditor could use t his to distinguish between what they are sure of and
> what should be checked by a senior auditor.
>
> My proposal would be to leave it in and to add a clear message in the report
> when not all criterion are answered with pass/fail/not present.
>
> Wilco
> ________________________________________
> Van: Gavin Evans [gavin.evans@digitalaccessibilitycentre.org]
> Verzonden: dinsdag 12 augustus 2014 9:30
> Aan: WebKeyIT
> CC: Shadi Abou-Zahra; Eval TF; Wilco Fiers
> Onderwerp: Re: WCAG-EM Report Tool
>
> Hi Vivienne/Shadi,
>
> So I just had a thought.
>
> Vivienne wrote:
> "That is why we have had to change our reporting and its something we argued
> back and forth with the group.  I agree with the idea (as did Detlev and
> others) that if it isn't there, how can it pass, but the above is pretty
> clear.  That it is isn't there, it is a pass"
>
> While I understand the clarity obtained from awarding a 'Pass' there are
> times when an evaluator/developer/web manager would be sometimes confused if
> they awarded a "pass" for a specific area and then, after evaluating the
> same page 6 months down the line and it failed.
>
> However the reason for this could well have been because the site had been
> updated within the last six months and implemented newer technology that may
> not have been tested.
>
> I think that there should be a clear indicator somewhere that, although
> "passes" did not contain any technology that conformed to the success
> criteria.
>
> Kindest Regards,
>
> Gavin Evans
> Director of Operations | DAC
> Mob: 07936 685804
> Twitter: @GavinAEvans @DACcessibility
> www.digitalaccessibilitycentre.org<http://www.digitalaccessibilitycentre.org
> />
> www.accessin.org<http://www.accessin.org/>
>
> On 12 Aug 2014, at 07:54, "WebKeyIT"
> <v.conway@webkeyit.com<mailto:v.conway@webkeyit.com>> wrote:
>
> Hi Shadi
>
> I thought that WCAG-EM was pretty clear that it could only be a pass or fail
> in accordance with the Conformance Requirements statement of WCAG 2.0 :
> http://www.w3.org/TR/WCAG20/#conformance-reqs:
>
>
>
> What does conformance mean?
>
> Conformance to a standard means that you meet or satisfy the 'requirements'
> of the standard. In WCAG 2.0 the 'requirements' are the Success Criteria. To
> conform to WCAG 2.0, you need to satisfy the Success Criteria, that is,
> there is no content which violates the Success Criteria.
>
>
> Note: This means that if there is no content to which a success criterion
> applies, the success criterion is satisfied.
>
> Most standards only have one level of conformance. In order to accommodate
> different situations that may require or allow greater levels of
> accessibility than others, WCAG 2.0 has three levels of conformance, and
> therefore, three levels of Success Criteria.
>
>
>
>
> That is why we have had to change our reporting and its something we argued
> back and forth with the group.  I agree with the idea (as did Detlev and
> others) that if it isn't there, how can it pass, but the above is pretty
> clear.  That it is isn't there, it is a pass.
>
> Regards,
>
> Vivienne Conway, B.IT (Hons), MACS CT, AALIA(CS) Director
>
> Web Key IT Pty Ltd
> PO BOX 681 Wanneroo, WA 6946
> M    0415 383 673   F   (08) 9325 6422
>
> E      v.conway@webkeyit.com<mailto:v.conway@webkeyit.com>
> W    www.webkeyit.com<http://www.webkeyit.com>
>
>
>
>
> This communication, including any attachments, is intended solely for the
> named addressee. It is confidential and may be subject to legal professional
> privilege. If you are not the intended recipient, please contact me
> immediately by reply email, delete it from your system and destroy any
> copies. This email is subject to copyright, no part of it should be
> reproduced, adapted or transmitted without the prior written consent of the
> copyright owner. Any views expressed in this message are those of the
> individual sender and may not necessarily reflect the views of Web Key IT
> Pty Ltd.
>
> -----Original Message-----
> From: Shadi Abou-Zahra [mailto:shadi@w3.org]
> Sent: Tuesday, 12 August 2014 2:18 PM
> To: WebKeyIT; 'Eval TF'
> Cc: 'Wilco Fiers'
> Subject: Re: WCAG-EM Report Tool
>
> Hi Vivienne,
>
> Good point. I think we need to change "inapplicable" to "not present" to
> match WCAG-EM terminology, and remove "cannot tell" from the list. Note that
> "untested" is not really a result. Maybe we should rename that to "not
> checked" or such to make it more apparent.
>
> Thanks,
>    Shadi
>
>
> On 12.8.2014 04:31, WebKeyIT wrote:
> HI Shadi
>
> In looking over the document, I see that one of the issues we faced with
> WCAG-EM is a problem here.  In the Audit Sample Tag (4), in the dropdown box
> for the 'results for the entire sample", there are options
> for:
> - untested
> - fail
> -pass
> -cannot tell
> -inapplicable
>
> We agreed in WCAG-EM that because WCAG does not allow anything except
> true/false, pass/fail that we could not allow these extra items into the
> report and still be able to say that WCAG-EM was used for the evaluation.
> This is something we have just gone through own reporting and removed as we
> had:
> -pass
> -fail
> -conditional pass (just for somethings that were small and not technically a
> failure)
> - n/a (for items not there such as multimedia)
> - nt (not tested) for items such as interruptions we could not be tested
> without access to onsite resources
>
> I would suggest that as we had to adjust WCAG-EM for just pass/fail, that
> this should also be changed in the WCAG-EM Reporter to be consistent.
>
>
> Regards,
>
> Vivienne Conway, B.IT (Hons), MACS CT, AALIA(CS) Director
>
> Web Key IT Pty Ltd
> PO BOX 681 Wanneroo, WA 6946
> M    0415 383 673   F   (08) 9325 6422
>
> E      v.conway@webkeyit.com<mailto:v.conway@webkeyit.com>
> W    www.webkeyit.com<http://www.webkeyit.com>
>
>
>
>
> This communication, including any attachments, is intended solely for the
> named addressee. It is confidential and may be subject to legal professional
> privilege. If you are not the intended recipient, please contact me
> immediately by reply email, delete it from your system and destroy any
> copies. This email is subject to copyright, no part of it should be
> reproduced, adapted or transmitted without the prior written consent of the
> copyright owner. Any views expressed in this message are those of the
> individual sender and may not necessarily reflect the views of Web Key IT
> Pty Ltd.
>
> -----Original Message-----
> From: Shadi Abou-Zahra [mailto:shadi@w3.org]
> Sent: Thursday, 31 July 2014 9:51 PM
> To: Eval TF
> Cc: Wilco Fiers
> Subject: WCAG-EM Report Tool
>
> Dear Eval TF,
>
> You may be interested to know about the "WCAG-EM Report Tool"
> currently being developed through the Education and Outreach Working Group
> (EOWG):
>    - http://w3c.github.io/wcag-em-report-tool/dist/
>
> This tool is currently an early prototype but we welcome your comments at
> this stage already. Please send comments to this list or preferably add them
> directly to the issues list on GitHub:
>    - https://github.com/w3c/wcag-em-report-tool/issues
>
> Let us know if you have any questions.
>
> Regards,
>     Shadi
>
> --
> Shadi Abou-Zahra - http://www.w3.org/People/shadi/ Activity Lead, W3C/WAI
> International Program Office Evaluation and Repair Tools Working Group (ERT
> WG) Research and Development Working Group (RDWG)
>
>
>
> --
> Shadi Abou-Zahra - http://www.w3.org/People/shadi/ Activity Lead, W3C/WAI
> International Program Office Evaluation and Repair Tools Working Group (ERT
> WG) Research and Development Working Group (RDWG)
>
>
>
>

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Received on Wednesday, 13 August 2014 06:03:07 UTC