Re: Task testing structure

John,

As Jon and I both pointed out: WCAG 2.x already allows, in a section of
Conformance marked "Optional"
<https://www.w3.org/TR/WCAG21/#conformance-claims>, content owners to
define the scope of their conformance claim. I've only tried swapping out
"web page" as the granularity of conformance with "task" so that the test
results relate more to the accessibility of what users want to do.

-Shawn

On Tue, Apr 28, 2020 at 8:28 AM John Foliot <john.foliot@deque.com> wrote:

> Jon writes:
>
> > If a site is perfectly accessible for ordering a pizza and looking up
> menus and buying gift cards but a game is not accessible does it really
> make sense to say to the site owner that you get no credit and you can’t
> claim conformance for the processes that do work?
>
> Absolutely not. Credit where credit is due, but to *exclude* the difficult
> or non-conforming part from the scoping, and thus "boosting" your score is
> also unacceptable. It's like saying *ALL my laundry is clean* (as long as
> you don't count the dirty pile...).
>
> I've long argued that the architecture of the scoring and scoping
> consideration was a crucial part of any solution, and repeatedly I've been
> shot-down or otherwise seen this discussion deferred. Scoping, as one of
> those concerns, needs to be clear and unambiguous, and I argue *should
> not* be set by the site owner in such a way as it allows them to exclude
> content.
>
>
> Shawn writes:
>
> > The NY Times with crossword puzzles as its own scope, separate from the
> actually-news-articles scope, probably makes a better example.
>
> ...and so? Does that now mean that the NY Times can 'exclude' the
> crossword puzzle from their conformance claim, as it was "scoped
> separately", and left out of the "legal" conformance claim because it has a
> very low score that would "drag down" the composite score of the NY Times?
> If you don't think multiple industries won't leverage that loop-hole to
> avoid their responsibilities, I'll suggest you are looking at things
> through rose-colored glasses.
>
> Any scoping scheme that allows content owners to segregate or exclude
> content from their conformance claim will be abused. A site's overall
> accessibility score *MUST* also include its weakest link, or we risk losing
> buy-in from the regulators (and let's for a minute be honest here: failing
> to get regulator buy-in will doom WCAG 3.x to sit on a shelf gathering
> dust, as so many other W3C technologies and specification have done so in
> the past.)
>
> JF
>
> On Mon, Apr 27, 2020 at 4:36 PM Jonathan Avila <jon.avila@levelaccess.com>
> wrote:
>
>>
>>    - The ability to cherry-pick what is and isn't out of scope is a
>>    dangerous precedent/concept, and will have (I fear) detrimental effects for
>>    persons with disabilities. Why wouldn't a "pizza game" be of interest to
>>    disabled users as well? Why shouldn't they also get to play along?
>>
>>
>>
>> Games and other aspects of sites are important to people with
>> disabilities – but scoping of content and prioritization of things to fix
>> is important.  If a site is perfectly accessible for ordering a pizza and
>> looking up menus and buying gift cards but a game is not accessible does it
>> really make sense to say to the site owner that you get no credit and you
>> can’t claim conformance for the processes that do work?  That is no
>> incentive and doesn’t take into account the practical reality of critical
>> paths.  It also doesn’t help the user understand and make decisions about
>> which sites to use and support.  Users with disabilities should have the
>> facts about what is conformant and then can make an informed decision to
>> use the site.  Regulators and courts can also then make informed
>> decisions.
>>
>>
>>
>> I’m not suggesting that people should cherry pick parts of pages or parts
>> of tasks – such as toppings for the pizza – but in complex sites it’s
>> important to be able to know and measure where the conformance problems are
>> – what parts of the page, what third party content, etc.  I also reinforce
>> that today conformance can be scoped by page – so this is already
>> possible.  I wonder if we can have two situations conformance calculation
>> by unit, workflows, etc. but not set some arbitrary minimum bar.  That is
>> conformance calculations based on unit and sum of units.   For example,
>> today we have WCAG but also the conformance model.   Also I would recommend
>> we keep non-interference requirements that acknowledge some issues on a
>> page can prevent any content on that page from being accessible.
>>
>>
>>
>> Jonathan
>>
>>
>>
>> *From:* John Foliot <john.foliot@deque.com>
>> *Sent:* Monday, April 27, 2020 2:27 PM
>> *To:* Shawn Lauriat <lauriat@google.com>; WCAG <w3c-wai-gl@w3.org>
>> *Cc:* Silver TF <public-silver@w3.org>
>> *Subject:* Re: Task testing structure
>>
>>
>>
>> *CAUTION:* This email originated from outside of the organization. Do
>> not click links or open attachments unless you recognize the sender and
>> know the content is safe.
>>
>>
>>
>> Hi Shawn,
>>
>>
>>
>> > Following that example of the pizza place site: they may have left a
>> pizza game out of their scope of conformance, judging it not a part of
>> their core offering of pizza. If someone has a problem with the pizza game
>> and raises that as an issue preventing them from getting pizza, everyone
>> then has the clear scope that left the game out, and whoever [was] involved
>> in the decision...
>>
>>
>>
>> ...now has to go to court and explain why they thought that game wasn't
>> important for disabled people?
>>
>>
>>
>> *Re: Scoping*
>>
>> The ability to cherry-pick what is and isn't out of scope is a
>> dangerous precedent/concept, and will have (I fear) detrimental effects for
>> persons with disabilities. Why wouldn't a "pizza game" be of interest to
>> disabled users as well? Why shouldn't they also get to play along? Because
>> making the pizza game accessible is too hard? - wrong answer...  (I recall
>> our colleague and friend Victor Tsaran once saying to me - and I paraphrase
>> - that today it's relatively easy to make sites 'accessible', but he could
>> hardly wait for the day when they were also "fun" - this from back when he
>> was still at Yahoo!, and they included an Easter Egg on the Yahoo! site:
>> https://youtu.be/xXNkP2jU7Pg)
>>
>>
>>
>> Selective Accessibility MUST be avoided, not encouraged, and I fear your
>> use-case is an example of why we shouldn't be leaving scoping to the
>> content owners (and also demonstrates how easy it will be for uninformed
>> content creators to miss the forest, because we've got them looking at -
>> and selecting - specific trees...) Using the same logic, I could also argue
>> that content in an <aside> isn't really critical to the main content (which
>> MUST be accessible) - that's why it is an aside - and so any content in an
>> <aside> is then out of scope? Slippery slope ahead.
>>
>>
>>
>> JF
>>
>>
>>
>> On Mon, Apr 27, 2020 at 12:36 PM Shawn Lauriat <lauriat@google.com>
>> wrote:
>>
>> Many good questions!
>>
>>
>>
>> I'm kinda liking aspects of this approach (ACT Rules format for testing
>> flows), but (of course) I have a critical question: *how do we score
>> something like this?*
>>
>>
>>
>> Honestly, I'd like to think through that as a separate thing to figure
>> out from the topic of scoping and task definition, though still heavily
>> related. We could end up with any number of scoring systems using the same
>> scoping and task definition. Trying to figure them out at the same time
>> just introduces too many variables for me.
>>
>>
>>
>> As I described it to Jeanne recently, I have this kind of thought about
>> how we could define scope and tasks, I have a clear-ish sense of how we can
>> build up tests for methods, but still have only murky ideas on how we can
>> get the two to meet in the middle. We've certainly made some good progress
>> on that, but we still definitely have further to go.
>>
>>
>>
>> Open question: is this a correct interpretation? Does all critical path
>> testing need to start from a common starting point?
>>
>>
>>
>> A really good question, and one that honestly depends on the site or app
>> (etc.). For a pizza site, you can link directly to the contact page. For an
>> app like Google Docs, you can't really link directly to text substitution
>> preferences, so that'd need to come from a more common start point. We
>> should help walk people through how to define and include this in scope,
>> definitely, as the accessibility of a thing doesn't really matter if you
>> can't get access to it in the first place.
>>
>>
>>
>> Additionally, how do we ensure that all critical path testing is scoped
>> by any given site? (the current scoping proposal leaves it to the
>> site-owner to scope their conformance claims, so leaving out complex or
>> critical flows that are non-conformant could be easily overcome by simply
>> leaving those flows out of the testing scope).
>>
>>
>>
>> I don't think we need to. If we have a clear definition of how to scope,
>> and a way for people to transparently declare that scope, we can leave the
>> "is this right?" part to those who need to decide it. Following that
>> example of the pizza place site: they may have left a pizza game out of
>> their scope of conformance, judging it not a part of their core offering of
>> pizza. If someone has a problem with the pizza game and raises that as an
>> issue preventing them from getting pizza, everyone then has the clear scope
>> that left the game out, and whoever involved in the decision as to whether
>> the scope should include the game can make it in an informed way.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Shawn
>>
>>
>>
>> On Mon, Apr 27, 2020 at 12:20 PM John Foliot <john.foliot@deque.com>
>> wrote:
>>
>> Hi Shawn
>>
>>
>>
>> I'm kinda liking aspects of this approach (ACT Rules format for testing
>> flows), but (of course) I have a critical question: *how do we score
>> something like this*?
>>
>> Each site(1) is going to have "critical paths" but few sites will be
>> sharing the same critical paths. Additionally, some paths or tasks (find
>> hours of operation) are significantly easier to do then others (update my
>> emergency contact information on my companies HR intranet), especially if
>> it pre-supposes that *all* paths start at a site's "homepage" (and/or the
>> outcome or solution to Success Criterion 2.4.5 Multiple Ways - i.e. a
>> sitemap page or search results page).
>>
>>
>>
>> No matter which, it seems to me that testing a critical path needs to
>> start *somewhere*, and for a scalable and repeatable testing regime, about
>> the only thing all sites have in common is a 'homepage', which is something
>> your example already suggests:
>>
>> 1.       Load the pizza restaurant's site
>>
>> 1.       Possible inputs: found via search engine, hit a bookmark link,
>> selected from browser's history, etc.
>>
>> 2.       *Main page loads* with focus at the top of the screen
>>
>> Open question: is this a correct interpretation? Does all critical path
>> testing need to start from a common starting point?
>>
>>
>>
>> Additionally, how do we ensure that *all *critical path testing is
>> scoped by any given site? (the current scoping proposal leaves it to the
>> site-owner to scope their conformance claims, so leaving out complex or
>> critical flows that are non-conformant could be easily overcome by simply
>> leaving those flows out of the testing scope).
>>
>>
>>
>> JF
>>
>>
>>
>> (1: site being an euphemism for 'online digital activity or presence' -
>> as we need to take XR and other emergent tech into account as well)
>>
>>
>>
>> On Mon, Apr 27, 2020 at 10:28 AM Shawn Lauriat <lauriat@google.com>
>> wrote:
>>
>> From an email I sent to some ACT folks a little while ago, where I had
>> tried expressing my thoughts on how we could use the same kind of structure
>> that ACT has, but as a way of essentially expressing overall scope as a set
>> of user journeys for task based testing. Hoping this can help for
>> tomorrow's conversation to have an example written out:
>>
>>
>>
>> For ACT rules, Link has accessible name
>> <https://act-rules.github.io/rules/c487ae> applies
>> <https://act-rules.github.io/rules/c487ae#applicability> to any HTML
>> element with the semantic role
>> <https://act-rules.github.io/rules/c487ae#semantic-role> of *link* that
>> is included in the accessibility tree
>> <https://act-rules.github.io/rules/c487ae#included-in-the-accessibility-tree>
>> . Link in context is <https://act-rules.github.io/rules/5effbb>
>> descriptive <https://act-rules.github.io/rules/5effbb> essentially
>> applies to any element that passes Link has accessible name
>> <https://act-rules.github.io/rules/c487ae>. In other words:
>>
>>    1. For each thing exposed in the accessibility tree as a link
>>
>>
>>    1. Go through Link has accessible name
>>       <https://act-rules.github.io/rules/c487ae> steps
>>       2. For each link that fails, note result
>>       3. For each link that passes
>>
>>
>>    1. Go through Link in context is
>>          <https://act-rules.github.io/rules/5effbb>descriptive
>>          <https://act-rules.github.io/rules/5effbb> steps
>>          2. For each link that fails, note result
>>
>> For tasks, even if simply in Education & Outreach type documentation, we
>> could walk people through the process of defining tasks and the steps
>> within each task similar to how the ACT Rules Format
>> <https://www.w3.org/TR/act-rules-format/> describes composite rules and
>> the atomic rules within each composite.
>>
>>
>>
>> The scope of a pizza restaurant's site could then have the definition of
>> a collection of tasks, the level at which they could/would measure overall
>> conformance:
>>
>>    1. Choose what kind of pizza to order from the available options
>>    2. Find out the hours of operation
>>    3. Find out how to get to the restaurant to dine in
>>    4. Contact the restaurant to order delivery
>>
>> Each task could consist of atomic actions, typically defined by design,
>> development, and testing activities. For task 2. Find out the hours of
>> operation, that could look like:
>>
>>    1. Load the pizza restaurant's site
>>
>>
>>    1. Possible inputs: found via search engine, hit a bookmark link,
>>       selected from browser's history, etc.
>>       2. Main page loads with focus at the top of the screen
>>
>>
>>    1. Navigate to contact page (composite, describes one possible path)
>>
>>
>>    1. Move focus to site navigation menu
>>       2. Open navigation menu
>>       3. Move focus to "Contact us" link
>>       4. Activate link
>>
>>
>>    1. Navigate to text containing the hours of operation (composite)
>>
>>
>>    1. Find "Hours of operation" section
>>       2. Read contents of "Hours of operation" section
>>
>> Within the steps of each atomic task bit, we could then run through the
>> applicability checks for each ACT-type Rule. So Link has accessible name
>> <https://act-rules.github.io/rules/c487ae> would apply to all links
>> within the path, but not to a random link in the footer that has a label
>> that doesn't imply any relation to hours or contact information.
>>
>>
>>
>> I have thoughts about how each of these could work and how we would
>> define applicability of rules and such based on the tasks, but I think it
>> would make sense to just start with this higher-level question of whether
>> we could (or should) have some kind of structured task definition similar
>> to ACT's current structured rule definition.
>>
>>
>>
>> -Shawn
>>
>>
>>
>>
>> --
>>
>> *​**John Foliot* | Principal Accessibility Strategist | W3C AC
>> Representative
>> Deque Systems - Accessibility for Good
>> deque.com
>>
>>
>>
>>
>>
>>
>>
>>
>> --
>>
>> *​**John Foliot* | Principal Accessibility Strategist | W3C AC
>> Representative
>> Deque Systems - Accessibility for Good
>> deque.com
>>
>>
>>
>>
>>
>
>
> --
> *​John Foliot* | Principal Accessibility Strategist | W3C AC
> Representative
> Deque Systems - Accessibility for Good
> deque.com
>
>
>