W3C home > Mailing lists > Public > public-wai-evaltf@w3.org > February 2012

AW: New version Methodology

From: Kerstin Probiesch <k.probiesch@googlemail.com>
Date: Thu, 9 Feb 2012 14:08:16 +0100
To: "'Shadi Abou-Zahra'" <shadi@w3.org>, "'Eric Velleman'" <evelleman@bartimeus.nl>
Cc: <public-wai-evaltf@w3.org>
Message-ID: <4f33c526.c77d0e0a.7414.47c6@mx.google.com>
Hi all,

sorry. I also disagree with item 9.

As I understood the discussion there wasn't an agreement about, that the
claim always sets the scope. As I remember the discussion we found some
cases for exceptions (Shadi mentioned). Item 9 is - as already mentioned in
the note - not conform with WCAG conformance statement. Which would mean
that an evaluation like this wouldn't be valid.

--Kerstin

 




> -----Ursprüngliche Nachricht-----
> Von: Shadi Abou-Zahra [mailto:shadi@w3.org]
> Gesendet: Donnerstag, 9. Februar 2012 13:58
> An: Eric Velleman
> Cc: public-wai-evaltf@w3.org
> Betreff: Re: New version Methodology
> 
> I also disagree with the addition to section 3 pointed out in item 9.
> 
> I think we can allow website owners to define what the target website
> is
> (an entire site, sub-site, application, portlet, etc.) but I think we
> agreed that the evaluation then applies to that entire website.
> 
> It should still be possible to *report* what parts of the website do
> not
> conform, but our methodology should not conflict with WCAG.
> 
> I think this also relates to item 3 about reporting Success Criteria
> met
> beyond the target level of conformance. The reporting section may be
> more key than currently assumed...
> 
> Best,
>    Shadi
> 
> 
> On 9.2.2012 13:15, Velleman, Eric wrote:
> > Hello Richard,
> >
> > In the discussion the general agreement was that the conformance
> claim sets the scope of the evaluation. This means that if a certain
> technology is chosen for the conformance claim, this could exclude
> parts of a complete process because the complete process is not what
> the evaluator would be looking at. If we say that all parts of the
> complete process should be in the evaluation this would limit the
> flexibility of the website owner to state the conformance claim for the
> evaluation.
> >
> > I do agree that in the current form, any conformance claim would end
> up being non conformant with WCAG2.0. This is why I added the question
> mark..
> >
> > Kindest regards,
> >
> > Eric
> > =========================
> > Eric Velleman
> > Technisch directeur
> > Stichting Accessibility
> > Universiteit Twente
> >
> > Oudenoord 325,
> > 3513EP Utrecht (The Netherlands);
> > Tel: +31 (0)30 - 2398270
> > www.accessibility.nl / www.wabcluster.org / www.econformance.eu /
> > www.game-accessibility.com/ www.eaccessplus.eu
> >
> > Lees onze disclaimer: www.accessibility.nl/algemeen/disclaimer
> > Accessibility is Member van het W3C
> > =========================
> >
> > ________________________________________
> > Van: RichardWarren [richard.warren@userite.com]
> > Verzonden: donderdag 9 februari 2012 13:00
> > Aan: Velleman, Eric; public-wai-evaltf@w3.org
> > Onderwerp: Re: New version Methodology
> >
> > Dear All,
> >
> > I am sorry - but I missed the discussions about excluding parts of a
> process
> > ( Point 9 in Eric's email). Surely this is the wrong way round. You
> cannot
> > exclude part of a **process**  because, by doing so, you then stop it
> being
> > a process.
> >
> > We must not confuse our terminology. It is possible to exclude parts
> of a
> > *website* from an evaluation because that is not a process that has
> to be
> > followed.
> >
> > You can evaluate individual parts of a process and say that "this
> part is
> > compliant" so that developers can make progress, but for the process
> itself
> > it is the whole process from beginning to end that must be compliant,
> > otherwise it is not a compliant process.
> >
> >
> > Richard
> >
> > -----Original Message-----
> > From: Velleman, Eric
> > Sent: Wednesday, February 08, 2012 11:39 PM
> > To: public-wai-evaltf@w3.org
> > Subject: New version Methodology
> >
> > Dear EvalTF,
> >
> > Tomorrow is our next Telco. I am looking forward to speaking to you
> all
> > after the many discussions in the last weeks. As an overview in this
> mail,
> > below are the changes I made to the new version. I only just
> delivered the
> > new version to Shadi, so I hope we can have a look at it during
> tomorrows
> > Telco.
> >
> > Date of new version: 20120209
> >
> > Changes to the document:
> >
> > 1. Because a sample only covers a small portion of a website and
> because we
> > want to evaluate conformance to WCAG 2.0, the entire sample should be
> > without failures of Success criteria. This means that any failure
> found
> > leads to non-conformance of the website regardless of the impact or
> barrier.
> > - Done
> >
> > 2. The section on error margin is deleted. This margin is now set to
> 0%.
> > - Done
> >
> > 3. Added the requirement in 6.1 that the conformance claim should
> provide a
> > “list of success criteria beyond the level of conformance claimed
> that have
> > been met. This information should be provided in a form that users
> can use,
> > preferably machine-readable metadata” (from:
> > http://www.w3.org/TR/WCAG/#conformance-claims).
> > - Done
> >
> > 4. We should include the possibility for website owners to fix
> incidental
> > errors without a totally new evaluation being necessary. Inserted
> into 5.5
> > - Done
> >
> > 5. Leonie adds to the previous: My suggestion is that after the
> website
> > owner has fixed the failed criteria, it isn't only the original
> sample of
> > pages that is tested. Instead it's a combination of pages from the
> original
> > sample and randomly selected new pages. Is that ok? Inserted into 5.5
> > - Done
> >
> > ------
> > Clause 3
> > ------
> >
> > 6. Significant changes to the scope section to reflect the
> discussion. Our
> > discussion makes the conformance claim the parent of the scope. This
> means
> > that the scope of an Evaluation is not necessarily the full website,
> but
> > flexible as discussed. This should then be more in line with the
> definition
> > of website that we plan to use.
> > - This means that the possibility to exclude web pages from the scope
> is no
> > longer required if the scope is set by the conformance claim as
> proposed
> > (clause 3). I took the text out. This section is only usefull if the
> > Methodology focuses on full websites (so it would be possible to make
> > exclusions for people who just want to evaluate a part of a website).
> > - Done
> >
> > 7. If scope is set by the conformance claim, then the following
> subclauses
> > are not longer necessary:
> > - Base URI (out)
> > - Key Functionalities (clause 3.4)(out)
> > - Perception and function (out)
> > - Alternative (out)
> > - Webpages behind authorization (out)
> > - Technologies used on the web pages (out)
> > - Dividing the scope into multiple evaluations (out)
> > All taken out because not necessary if the conformance claim is the
> parent
> > of the scope.
> > - Done
> >
> > 8. Complete processes are still necessary (in). I also kept the
> definitions
> > of resource and complete process in the document.
> > - Done
> >
> > 9. As requested by the group, I added (below the definitions in
> clause 3):
> > “It is possible to exclude particular sections of a website from the
> scope
> > even though they are part of a complete process. Examples for
> possible
> > exclusion are: user generated content, wiki’s, bulletin boards etc.“
> > @@@Note that we are less strict here than WCAG2.0 and conformance
> statements
> > made would not conform to WCAG2.0.
> > Also added: “The Evaluation can also focus only on specific
> technologies
> > excluding all other technologies used.”
> > @@@Note that this is also less strict than WCAG2.0 and would not
> conform
> > with WCAG conformance statement.
> > - Done
> >
> > ------
> > Clause 5
> > ------
> >
> > 10. Deleted subclause 5.5 Error Margin
> > - Done
> > ------
> > Clause 6
> > ------
> >
> > 11. Added requirement on conformance in 6.1 (are we sure we want
> that? It
> > would require to look at more than just the level claimed=more
> time=more
> > money)
> > - Done
> >
> > Kindest regards,
> >
> > Eric
> >
> >
> >
> >
> >
> >
> 
> --
> Shadi Abou-Zahra - http://www.w3.org/People/shadi/
> Activity Lead, W3C/WAI International Program Office
> Evaluation and Repair Tools Working Group (ERT WG)
> Research and Development Working Group (RDWG)
Received on Thursday, 9 February 2012 13:08:24 GMT

This archive was generated by hypermail 2.3.1 : Friday, 8 March 2013 15:52:13 GMT