Re: AW: Concerns about not covering full website

Hi Kerstin, 

No, I wouldn't call what you wrote provocative... 

From your email, you wrote you - "strongly disagree with a methodology with a scope that is externally defined in general", amongst other things...  

I'm really trying to get my head round what your saying, so I'd really like just to clarify something... Did you mean that:

In your opinion, the use of an external (e.g. third-party) evaluator is an essential part of a WCAG 2.0 Conformance evaluation (independently defining the scope, etc).  As, only a WCAG 2.0 Conformance claim based on WCAG 2.0 Conformance evaluation undertaken by an external evaluator could ever be fully trusted. 

All the best 

Alistair

On 29 Jan 2012, at 10:28, Kerstin Probiesch wrote:

> Hi Elle, all,
> 
> probably there is a way for some exceptions. But I strongly disagree with a
> methodology with a scope that is externally defined in general. Companies
> will always choose the most cheapest way for evaluating the accessibility of
> their websites or products. But later they want to sell their products and
> earn money with them. And I'm sure the same is true not only for the private
> sector. Even the public sector will seize every opportunity for reducing
> costs. The best way for doing so is: reducing the scope. What if the next
> website owner says: choose 3 "representative" pages. Than this is the scope.
> An evaluator will choose those pages and the company or any other website
> owner can publish a claim that according to this there is 100% conformance.
> Others will follow, cause it's very cheap, very fast. One can imagine a lot
> of different scenarios and I fear in the worst case we will come out with a
> methodology which depends on economic interests of companies. The question
> is also if a methodology like this evaluates WCAG2 or measure what companies
> and the public sector want to pay? Will there be a correlation with WCAG2 or
> merely a correlation with economic interests? Sorry for these probably
> provocative sentences, but I believe that every negative scenario which one
> can imagine will happen.
> 
> Anyway and as said before: I also recognize the necessity for some
> exceptions and for some guidelines for special cases but they must be linked
> to strict conditions. They could be placed in a separate chapter of our
> methodology:
> 
> # Follow-Up Checks / quality assurance (for website which are already
> evaluated according to our methodology)
> # Portions of website: Shops (as special and even typical cases for complete
> processes) 
> # and probably subdomains (but without permission placing the claim on any
> other subdomain or main page and with conditions like: a clear and unique
> separate content which also could be a standalone website)
> 
> Best
> 
> Kerstin  
> 
> 
> 
> 
> Von: Elle [mailto:nethermind@gmail.com] 
> Gesendet: Samstag, 28. Januar 2012 20:36
> An: Kerstin Probiesch
> Cc: RichardWarren; Alistair Garrison; Martijn Houtepen; Eval TF
> Betreff: Re: Concerns about not covering full website
> 
> I have an alternate example to consider:
> 
> A large e-Commerce website with several segmented portals catering to
> different audiences - This company is trying to secure a government contract
> to sell a specific product suite to the state of New Jersey. In order to
> secure this contract, the company must show accessibility conformance for
> the specific product suite portal.  
> 
> In our current model, the company would be forced to consider the entire
> e-Commerce website in its evaluation, which could be cost prohibitive and
> time consuming compared to the initial goal for securing the contract.
>  Additionally, it affects several different business teams within the
> company itself, so making the business case for evaluation (internal
> resources are always required) and remediation could be impossible.
> Additionally, the state of New Jersey may end up penalized, because
> companies find it an unreasonable burden to evaluate and change an entire
> enterprise website for a single contract. 
> 
> In an externally defined scope model, the company would identify with the
> state of New Jersey what the scope of the conformance claim needed to be,
> and likely include areas of the website that were equally utilized by all
> portals.  The company performs the evaluation, remediates the portal and any
> associated applications, and they win the contract. The result would be that
> a portion of the website is now compliant for anyone who uses that portal,
> and the company has a business case that can stand up on the profitability
> of building accessible websites.  Perhaps the next redesign of the entire
> e-Commerce website will consider it a core requirement to meet these
> standards.
> 
> 
> 
> Respectfully,
> Elle
> 
> 
> 
> On Sat, Jan 28, 2012 at 3:54 AM, Kerstin Probiesch
> <k.probiesch@googlemail.com> wrote:
> Hi all,
> 
> I agree with Martijn. The more I think about it I also feel that this
> approach is too flexible and I fear that in the end we will not come out
> with a harmonized evaluation methodology but with something we already have.
> 
> 
> Of course there are Organizations/Clients who invest in evaluations because
> they feel that it is important. But I believe the majority don't invest in
> that because of the users, because of the importance of accessibility. They
> invest because they have to and they will choose the most cheapest way for
> doing so and claiming conformance on the base of the less pages as possible.
> 
> I agree that we need some procedures for cases like:
> 
> #there was already an successful evaluation with a testing procedure which
> follows our methodology and a website owner included a shop. Will it be
> enough to evaluate just the shop? Does it depends on the time between the
> first evaluation and second of the shop? Are there other changes, other
> additional content?
> #fast growing websites
> 
> And probably some other well described exceptions: for example a website
> owner wants to claim just a shop. This should include the evaluation of the
> shop itself and the path to the shop.
> 
> These cases should be linked to strict defined conditions.
> 
> But I disagree with a methodology where a website owner (always) defines the
> scope. Except of course where a website owner don't want to claim anything.
> 
> Best
> 
> Kerstin
> 
> 
> Von: Elle [mailto:nethermind@gmail.com]
> Gesendet: Freitag, 27. Januar 2012 18:53
> An: RichardWarren
> Cc: Alistair Garrison; Martijn Houtepen; Eval TF
> Betreff: Re: Concerns about not covering full website
> 
> I also agree.  I've thought about this since our call from a few pragmatic
> and business angles. 
> 
> While I see the risks that Martin identified, I don't think people would
> invest in auditing and meeting conformance levels on individual pages just
> to spin it for better public opinion. Organizations are either committed to
> accessibility or they're forced to meet requirements due to regulatory or
> litigation reasons.  
> 
> Either way, that scope is defined by them, not W3C or a methodology. If
> organizations are committed to accessibility, we requiring a full website
> evaluation penalizes any phased efforts. Additionally, the concept of a
> "full website" is becoming less and less viable to companies in a component
> driven environment of content delivery. 
> 
> I do, however, want to request that we keep the URLs as a requirement in
> defining that scope (and not something smaller within a single page).
> 
> 
> Respectfully,
> Elle
> 
> On Fri, Jan 27, 2012 at 12:12 PM, RichardWarren <richard.warren@userite.com>
> wrote:
> Hi
> I agree with Alistair. We are supposed to be developing a methodology to
> help people deliver a reliable and trusted conformance claim (if
> appropriate). It is not our job to rewrite WCAG.
>  
> Richard
>  
> From: Alistair Garrison
> Sent: Friday, January 27, 2012 4:03 PM
> To: Martijn Houtepen ; Eval TF
> Subject: Re: Concerns about not covering full website
>  
> Hi Martijn,
>  
> I too feel we are giving people who want to evaluate conformity to WCAG 2.0
> a guide on how to do a proper, representative evaluation... the question I
> suppose is why must it be limited to only our definition of a website?
>  
> Can I just clarify something (it being late on a Friday) - are you saying
> that in your opinion the evaluation methodology should primarily support
> independent (e.g. third-party) evaluations of whole websites, like something
> you might need to support some sort of conformance scheme / badge?
>  
> I might be missing something, but I'd like to think that if I were a website
> owner I'd be able to check the claimed conformance of my own page, pages,
> sections, sub-domains or entire website using the W3C/WAI WCAG 2.0
> Evaluation methodology - and, without having to use a third-party. In fact,
> I think I would raise questions / eyebrows if I couldn't...
>  
> I suppose I would also like to think that if I verified my claimed
> conformance myself using this Evaluation methodology people would actually
> believe it - without the need to have it independently verified.  We should
> not forget, that there will be many folks out there who have a site but
> cannot afford to get it independently verified...
>  
> I'm certain that in this day-and-age a website owner would have more to
> loose, than gain, by using 'smart wording' to inflate their claims. I would
> even go so far as to suggest that the probability of a website owner making
> a WCAG 2.0 Conformance claim for a small part of their site and then
> implying its for the entire website (by saying or not saying) would be small
> - especially as the whole point of the WCAG 2.0 Conformance claim is to make
> this situation ever-so-plainly obvious...
>  
> Even so, you appear to have questions about the actual worth of WCAG 2.0
> Conformance claims (something you might consider raising with the WCAG 2.0
> working group).  Personally, I believe the WCAG 2.0 Conformance Claim
> concept to be quite sound... and, I can't really understand why (as members
> of a Task Force of the WCAG 2.0 WG) we wouldn't want to be encouraging their
> use...
>  
> The draft methodology is a draft, and as such not set in stone... I trust we
> hopefully have the possibility to change it if the consensus view is that
> the change (using a Conformance Claim to define our scope) is of value...
>  
> All the best...
>  
> Alistair
>  
> On 27 Jan 2012, at 14:55, Martijn Houtepen wrote:
> 
> 
> Hi Alistair, TF,
>  
> I think this approach will be getting a little bit too flexible. As I
> understand, all the work we have done until now is in order to evaluate
> whole websites. We want “a standardized way to evaluate the conformance of
> websites to WCAG 2.0.” (quoted from the introduction of our draft
> methodology), the whole draft document focuses on  ‘websites’ instead of web
> pages. I can understand from an owners point of view that he/she sometimes
> will want to split up a whole website into several subevaluations, but these
> subevaluations, in my opinion, will together comprise the entire website,
> and not leave out some part for any reason.
>  
> Some problems arise if we do follow this strategy:
>  
> If we only check what the owner of a website wants us to check, this creates
> the risk that an owner will only have the conforming parts of the website
> evaluated. An owner of a inaccessible website can then, with a little smart
> wording, proudly claim conformance (of f.e. a little subsection). An
> experienced user can differentiate between strong and weak claims, but a
> normal user can’t. This in turn creates two potential pitfalls: lots of
> ‘empty’ claims that in practice do not help users, f.e.: “Our homepage
> conforms to WCAG 2.0”, leaving out that none of the underlying pages
> conform.  Secondly the degradation of those claims that are truly useful to
> an user “Page X claims to conform (using an ‘empty’ claim) but I can’t use
> it, so I probably also can’t use page Y (that conforms and has a useful
> claim). Confusion as to what an evaluated conformance claim means, need to
> be avoided.
>  
> Secondly, this will create the possibility to exclude key scenario’s and
> complete paths. Again from the introduction: “The Methodology defines manual
> and semi-automated methods for selecting representative samples of web pages
> from websites that include complete processes.” I feel we are giving people
> who want to evaluate conformity to WCAG a guide on how to do a proper,
> representative evaluation of a website. If we only check pages supplied by a
> website owner we can not always include key scenario’s or complete paths. I
> think we do not want a web store claiming conformance, whilst it check-out
> procedure is inaccessible. Or YouTube claiming conformance except for the
> pages that have video’s. To form a statement about conformance, I feel we
> have to be able to independently evaluate the whole website including
> whatever it may contain. The resulting conformance statement, if made, will
> then be of practical use for people with disabilities, as they can trust a
> conformance claim to be of use for them.
>  
> Kind regards,
>  
> Martijn Houtepen
>  
>  
> ________________________________________
> Van: Alistair Garrison [mailto:alistair.j.garrison@gmail.com]
> Verzonden: vrijdag 27 januari 2012 10:31
> Aan: Eval TF
> CC: Eric Velleman; Detlev Fischer
> Onderwerp: Re: Concerns about not covering full website
>  
> Dear all,
>  
> Due to its many perceived benefits, I have long been a passionate advocate
> for using a website owner's WCAG 2.0 Conformance Claim as the scope of our
> evaluation - writing several emails in the past months about this very
> subject*.  The following summarises my thoughts to date...
>  
> I totally support the pros stated by both Detlev and Eric, emphasising that:
>  
> "This approach is very flexible and would make it possible for evaluators to
> verify a WCAG 2.0 claim conformance for more stable websites or fast growing
> websites, or portions of websites, sections of a website or even single
> pages in a website - using sampling methods for large websites as per our
> discussions."
>  
> In addition, I strongly believe that: 
>  
> 1) It makes evaluation more reproducible, and more efficient, as we know
> exactly what forms the scope of our evaluation (i.e. what is defined in the
> WCAG 2.0 Conformance claim at the time of evaluation, from which samples can
> be taken as necessary);
>  
> 2) Once an initial evaluation had been done (scope WCAG 2.0 Conformance
> claim) we would only ever need to assess new additions to the WCAG 2.0
> conformance claim on an on-going basis - hopefully saving people time and
> money;
>  
> 3) It tells the public exactly what conforms in the website - a bit more
> like the VPAT.  If the WCAG 2.0 conformance claim states, for example, that
> a website's shopping process (defined by urls) conforms to x and y,  a user
> should know it will be accessible to them;
>  
> 4) If the website contains any complete processes which back-ends into third
> party pages - we are in fact prevented from just saying "the whole website"
> conforms.  A WCAG 2.0 Conformance Claim, on the other hand, could be made
> for this website, and evaluated, if we choose to use WCAG 2.0 conformance
> claims as our scope.
>  
> 5) It supports, even promotes and encourages, detailed WCAG 2.0 conformance
> claims to be made by the people responsible for entire websites / parts of
> websites.
>  
> It would also mean that the majority of Section 3 could be re-defined simply
> e.g. "The scope of the evaluation is defined as all urls for which a WCAG
> 2.0 conformance claim is being made, at the time of the evaluation".
>  
> * Referenced emails from Nov / Dec 2011
>  
> http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Dec/0004.html
> http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Nov/0073.html
> http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Nov/0086.html
> http://lists.w3.org/Archives/Public/public-wai-evaltf/2011Dec/0023.html
>  
> All the best
>  
> Alistair
> On 27 Jan 2012, at 01:01, Velleman, Eric wrote:
> 
> Hi Detlev, all,
> 
> Yes, it was an interesting discussion this afternoon about the scope of an
> evaluation :-).
> 
> This afternoon we discussed the possibility to use the WCAG2.0 Evaluation
> Methodology not only for full website evaluation, but also for parts of
> websites. In the current version the evaluation focuses on the full website
> with a possibility to exclude parts from the scope.  If we want to include
> evaluating parts of a website (like only the WAI part of W3C or only the BAD
> website) this should be added in the scope section.
> 
> Today a different approach was proposed: We look at it the other way around:
> the Conformance claim determines the scope.
> 
> This approach is very flexible and makes it possible to claim conformance
> for portions of websites, possibly even for technologies or collections of
> pages or single pages. This makes the Methodology much more flexible for
> website owners who are not always interested in evaluation of the full
> website. This is a change to the requirements document but it will provide
> support for requests by website owners and evaluators wishing to just look
> at a specific part of a website.
> 
> I would propose that we do a short discussion this week on this approach. Do
> we leave the scope completely free for the site owner to decide?
> 
> Kindest regards,
> 
> Eric
> 
> 
> 
> ________________________________________
> Van: Detlev Fischer [fischer@dias.de]
> Verzonden: donderdag 26 januari 2012 17:16
> Aan: EVAL TF
> Onderwerp: Concerns about not covering full website
> 
> Hi everyone,
> 
> I think a lot of the heat in the disacussion we just had may be down to
> a misunderstanding.
> 
> Eric, no one says that the evaluation of an entire webiste is not a good
> idea or should not be done. If a site owner wants it done, that's fine.
> 
> If however, a site owner wants to look at a particular section (that can
> be many pages), it should be possible to evaluate just that. It's also a
> cost issue. Any conformance claim must make that limited scope perfectly
> clear, preferably by binding the evaluation to a set of URLs.
> 
> Increasingly, sites have many different sections, like user generated
> content. No sane person would claim even A-Level conformance for a wiki
> or bulletin board that allows unstructured text input. It would fail SC
> 1.3.1 in minutes or hours. That's why it is more straightforward to be
> able to include things even if they are important processes on a site.
> 
> Working out a sampling approach still makes sense even if you look at
> parts of a site - and also if you evaluate the whole site.
> 
> Regards,
> Detlev
>  
>  
> 
> 
> 
> 
> --
> If you want to build a ship, don't drum up the people to gather wood, divide
> the work, and give orders. Instead, teach them to yearn for the vast and
> endless sea.    
> - Antoine De Saint-Exupéry, The Little Prince
> 
> 
> 
> 
> 
> -- 
> If you want to build a ship, don't drum up the people to gather wood, divide
> the work, and give orders. Instead, teach them to yearn for the vast and
> endless sea.    
> - Antoine De Saint-Exupéry, The Little Prince
> 
> 
>

Received on Sunday, 29 January 2012 17:57:57 UTC