Re: Task testing structure

John,

In your previous straw-man example, a decision had been taken that the
> "pizza game" isn't part of the task of ordering a pizza, so you then argue
> it's out of scope.


I've offered that case as the pizza store owners arguing that it doesn't
need to go into scope, so we could talk through that conflict case. I
haven't made the case that things should remain out of scope for
accessibility, but that different scopes of accessibility should express
conformance.

And while I'll note that VPATS have a notion of "supporting with
> exceptions", and accept that we're going to see something similar with the
> Silver scoring methods, I personally will strenuously oppose selective
> scoping at the page or site level. There is a world of difference in saying
> "We've got an 85% Accessibility score, INCLUDING our non-accessible game"
> versus "We got to 85% conformance by REMOVING the game from our test
> scope".
>

Totally agreed for the desired end goal of complete coverage! I mean more
that for this increasingly strained example, the pizza place could have one
conformance claim for real-life-pizza-related tasks, and a second for the
pizza game.

The NY Times with crossword puzzles as its own scope, separate from the
actually-news-articles scope, probably makes a better example.

Jonathan put it quite well, I think. I've offered the word "task" for this,
while readily admitting that I don't like the term. I just haven't come
across a better thing to mean "the thing the user wants to do". Users don't
want to visit web pages, they want to read an article or play a game or
post a picture or something.

 Also I would recommend we keep non-interference requirements that
> acknowledge some issues on a page can prevent any content on that page from
> being accessible.


Definitely, and I want to make sure that this aspect addresses any risks to
element-level scoping. We've talked through a few ideas on this at the test
or guideline level, but looking forward to working through more tomorrow
(and beyond)!

-Shawn

On Mon, Apr 27, 2020 at 5:36 PM Jonathan Avila <jon.avila@levelaccess.com>
wrote:

>
>    - The ability to cherry-pick what is and isn't out of scope is a
>    dangerous precedent/concept, and will have (I fear) detrimental effects for
>    persons with disabilities. Why wouldn't a "pizza game" be of interest to
>    disabled users as well? Why shouldn't they also get to play along?
>
>
>
> Games and other aspects of sites are important to people with disabilities
> – but scoping of content and prioritization of things to fix is important.
> If a site is perfectly accessible for ordering a pizza and looking up menus
> and buying gift cards but a game is not accessible does it really make
> sense to say to the site owner that you get no credit and you can’t claim
> conformance for the processes that do work?  That is no incentive and
> doesn’t take into account the practical reality of critical paths.  It also
> doesn’t help the user understand and make decisions about which sites to
> use and support.  Users with disabilities should have the facts about what
> is conformant and then can make an informed decision to use the site.
> Regulators and courts can also then make informed decisions.
>
>
>
> I’m not suggesting that people should cherry pick parts of pages or parts
> of tasks – such as toppings for the pizza – but in complex sites it’s
> important to be able to know and measure where the conformance problems are
> – what parts of the page, what third party content, etc.  I also reinforce
> that today conformance can be scoped by page – so this is already
> possible.  I wonder if we can have two situations conformance calculation
> by unit, workflows, etc. but not set some arbitrary minimum bar.  That is
> conformance calculations based on unit and sum of units.   For example,
> today we have WCAG but also the conformance model.   Also I would recommend
> we keep non-interference requirements that acknowledge some issues on a
> page can prevent any content on that page from being accessible.
>
>
>
> Jonathan
>
>
>
> *From:* John Foliot <john.foliot@deque.com>
> *Sent:* Monday, April 27, 2020 2:27 PM
> *To:* Shawn Lauriat <lauriat@google.com>; WCAG <w3c-wai-gl@w3.org>
> *Cc:* Silver TF <public-silver@w3.org>
> *Subject:* Re: Task testing structure
>
>
>
> *CAUTION:* This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe.
>
>
>
> Hi Shawn,
>
>
>
> > Following that example of the pizza place site: they may have left a
> pizza game out of their scope of conformance, judging it not a part of
> their core offering of pizza. If someone has a problem with the pizza game
> and raises that as an issue preventing them from getting pizza, everyone
> then has the clear scope that left the game out, and whoever [was] involved
> in the decision...
>
>
>
> ...now has to go to court and explain why they thought that game wasn't
> important for disabled people?
>
>
>
> *Re: Scoping*
>
> The ability to cherry-pick what is and isn't out of scope is a
> dangerous precedent/concept, and will have (I fear) detrimental effects for
> persons with disabilities. Why wouldn't a "pizza game" be of interest to
> disabled users as well? Why shouldn't they also get to play along? Because
> making the pizza game accessible is too hard? - wrong answer...  (I recall
> our colleague and friend Victor Tsaran once saying to me - and I paraphrase
> - that today it's relatively easy to make sites 'accessible', but he could
> hardly wait for the day when they were also "fun" - this from back when he
> was still at Yahoo!, and they included an Easter Egg on the Yahoo! site:
> https://youtu.be/xXNkP2jU7Pg)
>
>
>
> Selective Accessibility MUST be avoided, not encouraged, and I fear your
> use-case is an example of why we shouldn't be leaving scoping to the
> content owners (and also demonstrates how easy it will be for uninformed
> content creators to miss the forest, because we've got them looking at -
> and selecting - specific trees...) Using the same logic, I could also argue
> that content in an <aside> isn't really critical to the main content (which
> MUST be accessible) - that's why it is an aside - and so any content in an
> <aside> is then out of scope? Slippery slope ahead.
>
>
>
> JF
>
>
>
> On Mon, Apr 27, 2020 at 12:36 PM Shawn Lauriat <lauriat@google.com> wrote:
>
> Many good questions!
>
>
>
> I'm kinda liking aspects of this approach (ACT Rules format for testing
> flows), but (of course) I have a critical question: *how do we score
> something like this?*
>
>
>
> Honestly, I'd like to think through that as a separate thing to figure out
> from the topic of scoping and task definition, though still heavily
> related. We could end up with any number of scoring systems using the same
> scoping and task definition. Trying to figure them out at the same time
> just introduces too many variables for me.
>
>
>
> As I described it to Jeanne recently, I have this kind of thought about
> how we could define scope and tasks, I have a clear-ish sense of how we can
> build up tests for methods, but still have only murky ideas on how we can
> get the two to meet in the middle. We've certainly made some good progress
> on that, but we still definitely have further to go.
>
>
>
> Open question: is this a correct interpretation? Does all critical path
> testing need to start from a common starting point?
>
>
>
> A really good question, and one that honestly depends on the site or app
> (etc.). For a pizza site, you can link directly to the contact page. For an
> app like Google Docs, you can't really link directly to text substitution
> preferences, so that'd need to come from a more common start point. We
> should help walk people through how to define and include this in scope,
> definitely, as the accessibility of a thing doesn't really matter if you
> can't get access to it in the first place.
>
>
>
> Additionally, how do we ensure that all critical path testing is scoped by
> any given site? (the current scoping proposal leaves it to the site-owner
> to scope their conformance claims, so leaving out complex or critical flows
> that are non-conformant could be easily overcome by simply leaving those
> flows out of the testing scope).
>
>
>
> I don't think we need to. If we have a clear definition of how to scope,
> and a way for people to transparently declare that scope, we can leave the
> "is this right?" part to those who need to decide it. Following that
> example of the pizza place site: they may have left a pizza game out of
> their scope of conformance, judging it not a part of their core offering of
> pizza. If someone has a problem with the pizza game and raises that as an
> issue preventing them from getting pizza, everyone then has the clear scope
> that left the game out, and whoever involved in the decision as to whether
> the scope should include the game can make it in an informed way.
>
>
>
> Thanks,
>
>
>
> Shawn
>
>
>
> On Mon, Apr 27, 2020 at 12:20 PM John Foliot <john.foliot@deque.com>
> wrote:
>
> Hi Shawn
>
>
>
> I'm kinda liking aspects of this approach (ACT Rules format for testing
> flows), but (of course) I have a critical question: *how do we score
> something like this*?
>
> Each site(1) is going to have "critical paths" but few sites will be
> sharing the same critical paths. Additionally, some paths or tasks (find
> hours of operation) are significantly easier to do then others (update my
> emergency contact information on my companies HR intranet), especially if
> it pre-supposes that *all* paths start at a site's "homepage" (and/or the
> outcome or solution to Success Criterion 2.4.5 Multiple Ways - i.e. a
> sitemap page or search results page).
>
>
>
> No matter which, it seems to me that testing a critical path needs to
> start *somewhere*, and for a scalable and repeatable testing regime, about
> the only thing all sites have in common is a 'homepage', which is something
> your example already suggests:
>
> 1.       Load the pizza restaurant's site
>
> 1.       Possible inputs: found via search engine, hit a bookmark link,
> selected from browser's history, etc.
>
> 2.       *Main page loads* with focus at the top of the screen
>
> Open question: is this a correct interpretation? Does all critical path
> testing need to start from a common starting point?
>
>
>
> Additionally, how do we ensure that *all *critical path testing is scoped
> by any given site? (the current scoping proposal leaves it to the
> site-owner to scope their conformance claims, so leaving out complex or
> critical flows that are non-conformant could be easily overcome by simply
> leaving those flows out of the testing scope).
>
>
>
> JF
>
>
>
> (1: site being an euphemism for 'online digital activity or presence' - as
> we need to take XR and other emergent tech into account as well)
>
>
>
> On Mon, Apr 27, 2020 at 10:28 AM Shawn Lauriat <lauriat@google.com> wrote:
>
> From an email I sent to some ACT folks a little while ago, where I had
> tried expressing my thoughts on how we could use the same kind of structure
> that ACT has, but as a way of essentially expressing overall scope as a set
> of user journeys for task based testing. Hoping this can help for
> tomorrow's conversation to have an example written out:
>
>
>
> For ACT rules, Link has accessible name
> <https://act-rules.github.io/rules/c487ae> applies
> <https://act-rules.github.io/rules/c487ae#applicability> to any HTML
> element with the semantic role
> <https://act-rules.github.io/rules/c487ae#semantic-role> of *link* that
> is included in the accessibility tree
> <https://act-rules.github.io/rules/c487ae#included-in-the-accessibility-tree>
> . Link in context is <https://act-rules.github.io/rules/5effbb>descriptive
> <https://act-rules.github.io/rules/5effbb> essentially applies to any
> element that passes Link has accessible name
> <https://act-rules.github.io/rules/c487ae>. In other words:
>
>    1. For each thing exposed in the accessibility tree as a link
>
>
>    1. Go through Link has accessible name
>       <https://act-rules.github.io/rules/c487ae> steps
>       2. For each link that fails, note result
>       3. For each link that passes
>
>
>    1. Go through Link in context is
>          <https://act-rules.github.io/rules/5effbb>descriptive
>          <https://act-rules.github.io/rules/5effbb> steps
>          2. For each link that fails, note result
>
> For tasks, even if simply in Education & Outreach type documentation, we
> could walk people through the process of defining tasks and the steps
> within each task similar to how the ACT Rules Format
> <https://www.w3.org/TR/act-rules-format/> describes composite rules and
> the atomic rules within each composite.
>
>
>
> The scope of a pizza restaurant's site could then have the definition of a
> collection of tasks, the level at which they could/would measure overall
> conformance:
>
>    1. Choose what kind of pizza to order from the available options
>    2. Find out the hours of operation
>    3. Find out how to get to the restaurant to dine in
>    4. Contact the restaurant to order delivery
>
> Each task could consist of atomic actions, typically defined by design,
> development, and testing activities. For task 2. Find out the hours of
> operation, that could look like:
>
>    1. Load the pizza restaurant's site
>
>
>    1. Possible inputs: found via search engine, hit a bookmark link,
>       selected from browser's history, etc.
>       2. Main page loads with focus at the top of the screen
>
>
>    1. Navigate to contact page (composite, describes one possible path)
>
>
>    1. Move focus to site navigation menu
>       2. Open navigation menu
>       3. Move focus to "Contact us" link
>       4. Activate link
>
>
>    1. Navigate to text containing the hours of operation (composite)
>
>
>    1. Find "Hours of operation" section
>       2. Read contents of "Hours of operation" section
>
> Within the steps of each atomic task bit, we could then run through the
> applicability checks for each ACT-type Rule. So Link has accessible name
> <https://act-rules.github.io/rules/c487ae> would apply to all links
> within the path, but not to a random link in the footer that has a label
> that doesn't imply any relation to hours or contact information.
>
>
>
> I have thoughts about how each of these could work and how we would define
> applicability of rules and such based on the tasks, but I think it would
> make sense to just start with this higher-level question of whether we
> could (or should) have some kind of structured task definition similar to
> ACT's current structured rule definition.
>
>
>
> -Shawn
>
>
>
>
> --
>
> *​**John Foliot* | Principal Accessibility Strategist | W3C AC
> Representative
> Deque Systems - Accessibility for Good
> deque.com
>
>
>
>
>
>
>
>
> --
>
> *​**John Foliot* | Principal Accessibility Strategist | W3C AC
> Representative
> Deque Systems - Accessibility for Good
> deque.com
>
>
>
>
>