Re: Non-text contrast research

Hello all,

First off, thank you to Alastair for kindly allowing us to share our work
with the WCAG community. I personally hope that, first, the work we do can
be useful to the larger WCAG community, and second, by incorporating the
feedback we get here, we can deliver stronger and more useful research in
general. To that end, I definitely appreciate the opportunity to receive
and address questions; please forgive the delay in doing so, though, since
there’s still an internal process we have to go through.

Speaking directly to the questions above:

Regarding replicating the study with those with cognitive impairments -- I
agree this would be an interesting next step for this line of research.
I’ll begin exploring this with others internally at Google, and will share
an update here when I can. Note, however: additional observations using the
protocol from this study won’t invalidate data already collected. The
research already completed, qualitative in nature, aims to identify signals,
not explicate prevalence.

Regarding adding a control group, timing responses, concern around
confounds -- This is currently being planned. Whereas the prior qualitative
study was intended to identify signals, I’m building a quantitative study
that will allow us to control for possible confounds. There are two primary
reasons we didn’t aim for a quantitative study to begin with: first, access
to and size of the target population for this initial study meant that we
would have had to spend much more time reaching an appropriate number of
participants an appropriate number of times to be able to show significance
of effect of design variation (recalling that our participants were from
our Trusted Tester program [1], more on that below). And second, insights
generated from qualitative work allow us to identify more foundational
concerns – e.g., rather than first focusing on the impact of varying
affordance contrast on a specific outcome measure (like time to click/tap),
we are able to highlight a plethora of other design factors that impact the
identifiability and perceived interactability of components. Follow-on
experimental work, as we’re planning above, can then aim to estimate the
size of the impact of those additional attributes.

Regarding concerns that results were intended to support existing Material
guidelines (e.g., bias) -- As Alastair pointed out above, outlines or
similar affordances aren’t required in WCAG guidance for buttons [2]; three
of four buttons described in the Material spec do have an outline [3]. The
goal of this study was intentionally scoped extremely narrowly to ensure
that we could adequately identify signals, focusing only on two questions:
varying only affordance contrast, are these components (1) identifiable,
and are they (2) perceived as interactable.

Regarding concern that study participants were expert users, and Googlers
-- None of the participants were Googlers. All of the participants were
from our Trusted Tester program [1]. Testers participating in the panel
aren’t limited by technical proficiency, so there are a broad range of
prior experiences and skill sets represented. If anyone is curious about
participating, please see the form at the bottom of the accessibility
initiatives page ([4]).

In general, regarding limitations and next steps -- It’s true that while we
haven’t addressed all the research questions with this study (e.g.,
separate experimental study, mentioned above), we are aiming to expand our
understanding of the design space, and how users encounter, perceive, and
interact with our designs. There’s a difference between trying to optimize
a single product for a discrete outcome (e.g., findability, operationalized
as time to click/tap) and trying to identify an area of expression where
the design of an arbitrary product is more usable and accessible. One of
the challenges of research for a design system rather than a product is
aiming to find balance between both of those goals.

All that said, I very much appreciate the feedback, and I’m looking forward
to working alongside everyone in this community moving forward!

Thanks,

-Michael

[1] https://www.google.com/accessibility/initiatives-research/

[2] https://www.w3.org/WAI/WCAG21/Understanding/non-text-contrast.html

[3] https://material.io/design/components/buttons.html#usage

[4]
https://docs.google.com/forms/d/e/1FAIpQLSfcb-l0mCZ__09SSyFAuI_k2WBLR05URYbR_Stv9N42u7GTiw/viewform


On Fri, May 17, 2019 at 4:48 PM Jonathan Avila <jon.avila@levelaccess.com>
wrote:

> My personal thoughts from reviewing the recording are that the goal seems
> to be to get research findings that support the material design tenants
> that borders are not needed and that other subtle affordances are
> acceptable because in the end after some amount of time the user can just
> figure it out.
>
>
>
> When people with low vision look at a page we may only see small parts of
> the page and rely on other visual factors such as borders and non-word
> indicators when we can’t read the text that is in our best vision..    We do
> this to find and locate items.   Searching for a word on a page is
> extremely difficult with low vision – that’s why control+f is so
> important.  There is also visual latency where users with visual
> impairments take time to find something and may miss something they should
> otherwise be able to see – but miss it the first time.   Making the user
> rely on reading all the words or wading through little font differences to
> eventually figure out that something is actionable is not realistic for
> real people with disabilities.  This study doesn’t address the needs of
> users with cognitive and learning disabilities and seems to focus on expert
> users who are employees of Google.  There are many other aspects of this
> case such as relying on users to mouse over something, etc.  that raise
> questions about the exact methods used.  While I applaud the effort to
> conduct research – I personally feel there are to many confounding
> variables and not enough attention paid to the amount of time and other
> disabilities that make the results of limited use.
>
>
>
> On many pages I resort to setting a large focus indicator with the Stylus
> extension and tabbing around and also setting a border on actionable
> elements via hover and mousing around to try and determine actionable
> elements on a page.
>
>
>
> For me lack of borders causes the issue of not knowing the hit area for a
> target unless the mouse changes and then if there are two elements close by
> I’m not certain of which one is being targeted.  Borders provides
> confidence that I am within the hit area.   Lack of borders also is
> confusing regarding how things are related or grouped as it may be unclear
> what items are in the group or the type of action.  For example, the word
> “yes” by itself could be a yes radio button or a “yes” button.  One is
> checkable while the other performs some action that may take me somewhere
> else.  Having borders like square and circles gives me some affordance to
> know the type of response that will occur.  These are important aspects
> that must be considered.
>
>
>
> Jonathan
>
> .
>
>
>
> *From:* Alastair Campbell <acampbell@nomensa.com>
> *Sent:* Friday, May 17, 2019 11:39 AM
> *To:* W3C WAI ig <w3c-wai-ig@w3.org>
> *Subject:* RE: Non-text contrast research
>
>
>
> *CAUTION:* This email originated from outside of the organization. Do not
> click links or open attachments unless you recognize the sender and know
> the content is safe.
>
>
>
> Hi Everyone,
>
>
>
> Hopefully people will get a chance to review the slides and/or video I
> posted from Michael Gilbert and the team at Google [1]. Michael is now on
> this email group so can join in.
>
>
>
> I thought I start the comments with what I took way from the results:
>
>
>
>    - The structure of the criteria text gives us some flexibility, where
>    it says “Visual information *required* to identify user interface
>    components and states”, if research finds that X, Y & Z other factors make
>    the contrast irrelevant in a particular scenario that can be addressed
>    fairly easily.
>
>    That is already the case for buttons where the understanding doc [2]
>    says buttons don’t require borders.
>    - The remit of the guidelines is to prevent barriers that affect
>    people with disabilities, it would be useful to have a control group or a
>    comparison with other usability testing to help work out which factors
>    impact people with low vision, compared to a general audience. (Not that it
>    is a deciding factor, but it’s part of the equation.)
>    - I fully appreciate that more examples would help, but to make that a
>    manageable task it would help to know which types of component people have
>    struggled to apply the criteria to. Presumably the examples in the
>    understanding document [2] cover some cases, which other components are
>    people concerned with?
>    - This criteria (non-text contrast) is focused on having contrast for
>    certain aspects, but it does not require particular design
>    approaches/affordances. E.g. if an input doesn’t have any border it isn’t
>    required to have a contrasting one.
>    However, lack of affordance *is an issue* for many folk (particularly
>    with cognitive impairments [3]), it would be great to re-run the study with
>    participants with cognitive impairments.
>
> I’d just note that my brain is fairly wired-up to how the guidelines work,
> so I hope people less biased by that can comment as well 😊
>
>
>
> Kind regards,
>
>
>
> -Alastair
>
>
>
> 1] Page with video and link to slides:
>
> https://alastairc.uk/tests/wcag21-examples/ntc-research-video.html
>
> NB: If the slides don’t work in your screenreader make sure the
> accessibility setting is on:
>
> https://support.google.com/docs/answer/6282736
>
>
>
> 2] Understanding doc:
> https://www.w3.org/WAI/WCAG21/Understanding/non-text-contrast.html
>
>
>
> 3] COGA doc:
>
> https://w3c.github.io/coga/techniques/index.html#use-clear-visual-affordances
>
>
>
> --
>
>
>
> www.nomensa.com / @alastc
>


-- 
Michael Gilbert |  Senior UX Researcher  | mdgilbert@google.com | go/mxr