Re: WCAG-EM Report Tool

Hi Shadi,

My humble apologise - I did not read the email heading correctly. YES this 
is about the REPORTING tool not EM.

So I agree that something is required that enables a report to indicate that 
something has not (yet) been tested.

Sorry

Regards

Richard

-----Original Message----- 
From: Shadi Abou-Zahra
Sent: Wednesday, August 13, 2014 7:02 AM
To: Wilco Fiers ; Richard ; ryladog@gmail.com ; emmanuelle@sidar.org ; 
gavin.evans@digitalaccessibilitycentre.org ; v.conway@webkeyit.com
Cc: public-wai-evaltf@w3.org
Subject: Re: WCAG-EM Report Tool

Hi Richard, all,

Also note that this is a report generator tool based on WCAG-EM, not a
WCAG-EM report validator. Of course we want to help people to create
valid WCAG-EM reports but the tool has broader uses. Also, as Wilco
pointed out, there may be times when the report is temporarily not
meeting the WCAG-EM requirements. We have the "indicator" added to the
issues list (issue #73):
  - https://github.com/w3c/wcag-em-report-tool/issues

Best,
   Shadi


On 13.8.2014 01:19, Wilco Fiers wrote:
> Hi Richard,
> Yes that is actually what it would be used for, to indicator, to indicate 
> that the results aren't complete yet. Everything starts as untested, and 
> when you're done everything is set to passed/failed/not present. You are 
> right that a completed audit wouldn't have anything set to not checked.
>
> Wilco
>
>
> -------- Oorspronkelijk bericht --------
> Van: Richard <richard.warren@userite.com>
> Datum:12-08-2014 18:39 (GMT+01:00)
> Aan: ryladog@gmail.com, emmanuelle@sidar.org, Wilco Fiers 
> <w.fiers@accessibility.nl>, gavin.evans@digitalaccessibilitycentre.org, 
> v.conway@webkeyit.com
> Cc: shadi@w3.org, public-wai-evaltf@w3.org
> Onderwerp: RE: WCAG-EM Report Tool
>
> Sorry to be a pain but we cannot use terms like "cannot tell" or "not 
> tested". These terms indicate that the site was not proprly checkrd and 
> therefore makes the result meaningless.
>
> The only alternative to pass/fail is "not present " (which is. in effect. 
> a pass as there is no barrier created)
>
> Richard (by the pool)
>
> Sent from Samsung tablet
>
>
>
> Katie Haritos-Shea GMAIL <ryladog@gmail.com> wrote:
>
>
> +1
>
>
>
> * katie *
>
> Katie Haritos-Shea
> Senior Accessibility SME (WCAG/Section 508/ADA/AODA)
>
> Cell: 703-371-5545 | ryladog@gmail.com | Oakton, VA | LinkedIn Profile |
> Office: 703-371-5545
>
> -----Original Message-----
> From: Emmanuelle Gutiérrez y Restrepo [mailto:emmanuelle@sidar.org]
> Sent: Tuesday, August 12, 2014 7:04 AM
> To: 'Wilco Fiers'; 'Gavin Evans'; 'WebKeyIT'
> Cc: 'Shadi Abou-Zahra'; 'Eval TF'
> Subject: RE: WCAG-EM Report Tool
>
> I agree with Wilco.
>
> "Cannot tell" is very useful.
>
> Best,
>
> Emmanuelle Gutiérrez y Restrepo
> Patrono y Directora General
> Fundación Sidar - Acceso Universal
> Email: coordina@sidar.org
> Personal: Emmanuelle@sidar.org
> Web: http://sidar.org
>
>
> -----Mensaje original-----
> De: Wilco Fiers [mailto:w.fiers@accessibility.nl] Enviado el: martes, 12 
> de
> agosto de 2014 10:23
> Para: Gavin Evans; WebKeyIT
> CC: Shadi Abou-Zahra; Eval TF
> Asunto: RE: WCAG-EM Report Tool
>
> Hi everyone,
>
> Interesting discussion! I've also had the following issue suggested on
> Github:
> https://github.com/w3c/wcag-em-report-tool/issues/69
>
> Renaming Untested to Not checked and Inapplicable to Not present seems 
> like
> a good idea to me. I don't think Cannot tell should be removed though. Of
> course in the final report, if there is some criterion set to Cannot tell
> then the website wasn't fully audited, which should be clearly indicated.
> But there are quite a few scenarios in which a criterion can be set to
> Cannot tell. The main one I think would be if initial data was generated 
> by
> an external source. Tools that test part of a criterion shouldn't report
> that criterion as passed or failed, but as failed or cannot tell. And
> because Cannot tell is part of EARL it's quite plausible that tools will
> provide this result.
>
> If nothing else, at least it should be shown if this is the case. Another
> use case would be when multiple evaluators work on the same audit. A 
> junior
> auditor could use t his to distinguish between what they are sure of and
> what should be checked by a senior auditor.
>
> My proposal would be to leave it in and to add a clear message in the 
> report
> when not all criterion are answered with pass/fail/not present.
>
> Wilco
> ________________________________________
> Van: Gavin Evans [gavin.evans@digitalaccessibilitycentre.org]
> Verzonden: dinsdag 12 augustus 2014 9:30
> Aan: WebKeyIT
> CC: Shadi Abou-Zahra; Eval TF; Wilco Fiers
> Onderwerp: Re: WCAG-EM Report Tool
>
> Hi Vivienne/Shadi,
>
> So I just had a thought.
>
> Vivienne wrote:
> "That is why we have had to change our reporting and its something we 
> argued
> back and forth with the group.  I agree with the idea (as did Detlev and
> others) that if it isn't there, how can it pass, but the above is pretty
> clear.  That it is isn't there, it is a pass"
>
> While I understand the clarity obtained from awarding a 'Pass' there are
> times when an evaluator/developer/web manager would be sometimes confused 
> if
> they awarded a "pass" for a specific area and then, after evaluating the
> same page 6 months down the line and it failed.
>
> However the reason for this could well have been because the site had been
> updated within the last six months and implemented newer technology that 
> may
> not have been tested.
>
> I think that there should be a clear indicator somewhere that, although
> "passes" did not contain any technology that conformed to the success
> criteria.
>
> Kindest Regards,
>
> Gavin Evans
> Director of Operations | DAC
> Mob: 07936 685804
> Twitter: @GavinAEvans @DACcessibility
> www.digitalaccessibilitycentre.org<http://www.digitalaccessibilitycentre.org
> />
> www.accessin.org<http://www.accessin.org/>
>
> On 12 Aug 2014, at 07:54, "WebKeyIT"
> <v.conway@webkeyit.com<mailto:v.conway@webkeyit.com>> wrote:
>
> Hi Shadi
>
> I thought that WCAG-EM was pretty clear that it could only be a pass or 
> fail
> in accordance with the Conformance Requirements statement of WCAG 2.0 :
> http://www.w3.org/TR/WCAG20/#conformance-reqs:
>
>
>
> What does conformance mean?
>
> Conformance to a standard means that you meet or satisfy the 
> 'requirements'
> of the standard. In WCAG 2.0 the 'requirements' are the Success Criteria. 
> To
> conform to WCAG 2.0, you need to satisfy the Success Criteria, that is,
> there is no content which violates the Success Criteria.
>
>
> Note: This means that if there is no content to which a success criterion
> applies, the success criterion is satisfied.
>
> Most standards only have one level of conformance. In order to accommodate
> different situations that may require or allow greater levels of
> accessibility than others, WCAG 2.0 has three levels of conformance, and
> therefore, three levels of Success Criteria.
>
>
>
>
> That is why we have had to change our reporting and its something we 
> argued
> back and forth with the group.  I agree with the idea (as did Detlev and
> others) that if it isn't there, how can it pass, but the above is pretty
> clear.  That it is isn't there, it is a pass.
>
> Regards,
>
> Vivienne Conway, B.IT (Hons), MACS CT, AALIA(CS) Director
>
> Web Key IT Pty Ltd
> PO BOX 681 Wanneroo, WA 6946
> M    0415 383 673   F   (08) 9325 6422
>
> E      v.conway@webkeyit.com<mailto:v.conway@webkeyit.com>
> W    www.webkeyit.com<http://www.webkeyit.com>
>
>
>
>
> This communication, including any attachments, is intended solely for the
> named addressee. It is confidential and may be subject to legal 
> professional
> privilege. If you are not the intended recipient, please contact me
> immediately by reply email, delete it from your system and destroy any
> copies. This email is subject to copyright, no part of it should be
> reproduced, adapted or transmitted without the prior written consent of 
> the
> copyright owner. Any views expressed in this message are those of the
> individual sender and may not necessarily reflect the views of Web Key IT
> Pty Ltd.
>
> -----Original Message-----
> From: Shadi Abou-Zahra [mailto:shadi@w3.org]
> Sent: Tuesday, 12 August 2014 2:18 PM
> To: WebKeyIT; 'Eval TF'
> Cc: 'Wilco Fiers'
> Subject: Re: WCAG-EM Report Tool
>
> Hi Vivienne,
>
> Good point. I think we need to change "inapplicable" to "not present" to
> match WCAG-EM terminology, and remove "cannot tell" from the list. Note 
> that
> "untested" is not really a result. Maybe we should rename that to "not
> checked" or such to make it more apparent.
>
> Thanks,
>    Shadi
>
>
> On 12.8.2014 04:31, WebKeyIT wrote:
> HI Shadi
>
> In looking over the document, I see that one of the issues we faced with
> WCAG-EM is a problem here.  In the Audit Sample Tag (4), in the dropdown 
> box
> for the 'results for the entire sample", there are options
> for:
> - untested
> - fail
> -pass
> -cannot tell
> -inapplicable
>
> We agreed in WCAG-EM that because WCAG does not allow anything except
> true/false, pass/fail that we could not allow these extra items into the
> report and still be able to say that WCAG-EM was used for the evaluation.
> This is something we have just gone through own reporting and removed as 
> we
> had:
> -pass
> -fail
> -conditional pass (just for somethings that were small and not technically 
> a
> failure)
> - n/a (for items not there such as multimedia)
> - nt (not tested) for items such as interruptions we could not be tested
> without access to onsite resources
>
> I would suggest that as we had to adjust WCAG-EM for just pass/fail, that
> this should also be changed in the WCAG-EM Reporter to be consistent.
>
>
> Regards,
>
> Vivienne Conway, B.IT (Hons), MACS CT, AALIA(CS) Director
>
> Web Key IT Pty Ltd
> PO BOX 681 Wanneroo, WA 6946
> M    0415 383 673   F   (08) 9325 6422
>
> E      v.conway@webkeyit.com<mailto:v.conway@webkeyit.com>
> W    www.webkeyit.com<http://www.webkeyit.com>
>
>
>
>
> This communication, including any attachments, is intended solely for the
> named addressee. It is confidential and may be subject to legal 
> professional
> privilege. If you are not the intended recipient, please contact me
> immediately by reply email, delete it from your system and destroy any
> copies. This email is subject to copyright, no part of it should be
> reproduced, adapted or transmitted without the prior written consent of 
> the
> copyright owner. Any views expressed in this message are those of the
> individual sender and may not necessarily reflect the views of Web Key IT
> Pty Ltd.
>
> -----Original Message-----
> From: Shadi Abou-Zahra [mailto:shadi@w3.org]
> Sent: Thursday, 31 July 2014 9:51 PM
> To: Eval TF
> Cc: Wilco Fiers
> Subject: WCAG-EM Report Tool
>
> Dear Eval TF,
>
> You may be interested to know about the "WCAG-EM Report Tool"
> currently being developed through the Education and Outreach Working Group
> (EOWG):
>    - http://w3c.github.io/wcag-em-report-tool/dist/
>
> This tool is currently an early prototype but we welcome your comments at
> this stage already. Please send comments to this list or preferably add 
> them
> directly to the issues list on GitHub:
>    - https://github.com/w3c/wcag-em-report-tool/issues
>
> Let us know if you have any questions.
>
> Regards,
>     Shadi
>
> --
> Shadi Abou-Zahra - http://www.w3.org/People/shadi/ Activity Lead, W3C/WAI
> International Program Office Evaluation and Repair Tools Working Group 
> (ERT
> WG) Research and Development Working Group (RDWG)
>
>
>
> --
> Shadi Abou-Zahra - http://www.w3.org/People/shadi/ Activity Lead, W3C/WAI
> International Program Office Evaluation and Repair Tools Working Group 
> (ERT
> WG) Research and Development Working Group (RDWG)
>
>
>
>

-- 
Shadi Abou-Zahra - http://www.w3.org/People/shadi/
Activity Lead, W3C/WAI International Program Office
Evaluation and Repair Tools Working Group (ERT WG)
Research and Development Working Group (RDWG)

Richard Warren
Technical Manager
Website Auditing Limited (Userite)
http://www.userite.com

Received on Monday, 18 August 2014 10:45:03 UTC