Usability Test Plan

Thanks Charlotte and the Visa folks for this! The plan looks great! I am very excited for the usability testing!

Attached is a Word doc with a few comments tracked as changes -- for example, some content that won't be in the prototype, and suggestion for standards/guidelines tasks. Please feel free to call me if you have any questions or want more info: 617-395-7664

(Please send me a quick reply to let me know that you received this.)

Thanks!
~Shawn

On 9/20/2017 1:07 PM, Eric Eggert wrote:
> FYI
> 
> 
> 
> --
> 
> Eric Eggert
> Web Accessibility Specialist
> Web Accessibility Initiative (WAI) at World Wide Web Consortium (W3C)
> 
> 
> Forwarded message:
> 
>     From: Wise, Charlotte <cwise@visa.com>
>     To: Sharron Rush <srush@knowbility.org>, Green, James <jgreen@visa.com>, Bakken, Brent <brent.bakken@pearson.com>, Sharron Rush <srush@knowbility.org>, Eric Eggert <ee@w3.org>
>     Subject: Agenda item for today's meeting - review test questions
>     Date: Wed, 20 Sep 2017 16:43:19 +0000
> 
>     Hi all,
> 
>     Here are the general questions we plan to ask during the usability testing the first week of October. I’d like to discuss during this week’s call as the test plan will be finalized and tested internally next Thursday.
> 
> 
>         OBJECTIVES
> 
>     ·How do people interact with the WAI web site? Does the home page convey the breadth and scope of the resources WAI provides?
> 
>     ·Is the information architecture for the site clear, navigable, and does it assist users from different audiences in finding the content they seek?
> 
>     ·Is the visual design and page layout aesthetically pleasing to visitors?
> 
>     ·Does the content provided on the site make sense, meet the needs of different audiences, and establish WAI as a credible accessibility resource?
> 
> 
>         Intro Questions
> 
>     1.Tell me a little bit about your current job. Where do you work and what is your role?
> 
>     2.Can you describe for me what accessibility is in your own words?
> 
>     3.Do you have any experience with accessibility?
> 
>     4.Can you describe how accessibility is relevant to your work, or how it might impact your work in your current position?
> 
>     5.What resources do you use to get answers when you have accessibility-related questions?
> 
>     oWhat makes this resource useful?
> 
>     oWhat do you get out of it?
> 
>     6.Let’s say you needed to solve an accessibility problem for a project. Walk me through how you would go about finding that information.
> 
>     Task 1 – First Impressions
>     Goal: Gauge how effective the homepage is in communicating information to users **
> 
>     *Task:*  Take a look at the homepage of the WAI website.
> 
>     ·Q1: Have you visited this site before?**
> 
>     oWhere do you typically go to when you visit this site? **
> 
>     ·Q2: What is your first impression here?**
> 
>     oWhat do you think you can do on this site? **
> 
>     ·Q3: Before we begin our tasks, let’s take a look at the navigation briefly. What kinds of information do you think the different section headings would contain?  [Walk participants through each section and probe on specific questions noted]**
> 
>     oWhere do you think you would go to find Tutorials? (It’s under Design & Develop)**
> 
>     oAccessibility Fundamentals – probe on **
> 
>     §What do you think “standards harmonization” mean? **
> 
>     §What do you think “diverse abilities and barriers” means in relation to people with disabilities? (nested under How People With Disabilities Use The Web)**
> 
>     oPlan & Manage – probe on**
> 
>     §What do you think “Approaches for Interim Repairs?” mean?**
> 
>     oTest & Evaluate – what does “Using Combined Expertise” mean to you?**
> 
>     ·Q4: How can you tell if a section in the secondary navigation has additional content?**
> 
>     ·Q5: Which sections interest you the most or would be the most relevant to your work? **
> 
>     Task 2 – Get Started with Accessibility
>     Goal: Determine if users can easily find and understand general information on accessibility **
> 
>     Scenario: You’re working on a website for work and you’ve been told you need to make it accessible, but you don’t know where to start or what that really means.
> 
>     *Task*: Where would you go to find information on getting started on accessibility?
> 
>     ·Q6: What kind of information do you see here? How do you feel about this overview? **
> 
>     ·Q7: How do you feel about the amount of information on this page? **
> 
>     ·Q8: After reading this, what do you need to do get started with web accessibility? **
> 
>     ·Q9: How would you define what accessibility is in your own words?**
> 
>     ·**
> 
>     */Task 3 – Building a Business Case
>     /*/Goal: Determine if users can easily find information supporting the business case for accessibility**/*//*
> 
>     *Scenario: *You’re having trouble convincing your boss to make accessibility a priority for an app that your team is building and are wondering if WAI has any information that could help you make the case for him.
> 
>     *Task: *Where should you go to find information to help you convince your boss?
> 
>     ·*Q10: *What kind of information do you see here? **
> 
>     ·*Q11: *How do you feel about the amount of information?****
> 
>     ·*Q12: *From this page, where would you go to find financial arguments for prioritizing accessibility? /(//Path is WAI Home //à//Getting Started //à//Business Case //à//Financial Factors) **/
> 
>     ·*Q13: *On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task? //
> 
>     */Task 4 – Legal information
>     /*/Goal: Determine if users can easily find legal information related to accessibility /*//*
> 
>     *Scenario: *Your company is starting to do a lot more work with a partner firm in Mexico. You want to know what the laws are in Mexico concerning accessibility so you can determine how that might affect your company.
> 
>     *Task: *Where would you go to find this information?
> 
>     ·*Q14: *What kind of information do you see here?****
> 
>     ·*Q15:*How accurate do you think this information is? **
> 
>     ·*Q16:*How relevant do you think this information is?**
> 
>     ·*Q17: *How do you feel about the amount of information?****
> 
>     ·*Q18: *On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task? **
> 
>     **
> 
>     */Task 5 –  Developing an Accessibility Policy
>     /*/Goal: Determine if users can easily find information on developing an accessibility policy for their organization. ///
> 
>     *Scenario: *Your company has recently begun considering their approach to accessibility. Your manager has asked you to do some research on developing an accessibility policy for your organization.
> 
>     *Task: *Where we would you look to find this information?
> 
>     ·*Q19: *What kind of information do you see here? **
> 
>     ·*Q20: *How helpful do you think you would find this page in writing an accessibility policy? ****
> 
>     ·*Q21: *On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task?
> 
>     **
> 
>     Task 6 – *Easy Checks *
>     Goal: Determine if users can easily find and understand how to begin testing for accessibility **
> 
>     *Scenario*: Let's say you want information on how to get started assessing** the accessibility**of a web page.
> 
>     *Task*: Where would you go to find information on get started assessing the accessibility of a web page?
> 
>     ·Q22: What do you think the term “Easy Checks” means? **
> 
>     ·Q23: What kind of information do you see here? **
> 
>     ·Q24: Based on what you see on this page, what are Easy Checks? **
> 
>     ·Q25: How useful is this information in helping you get started on assessing accessibility? **
> 
>     ·Q26: On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task? **
> 
>     **
> 
>     Task 7 –Users with Specific disabilities
>     Goal: Determine if users can easily find and understand information on people with specific disabilities **
> 
>     *Scenario*: You are interested in learning more about how users with deafness or hearing disabilities experience the web.
> 
>     *Task*: Where would you go to find information on how users with hearing disabilities experience the web?
> 
>     ·*Q27: *What kind of information do you see here? How do you feel about this overview? **
> 
>     ·*Q28:*How do you feel about the amount of information? **
> 
>     ·*Q29*: On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task? **
> 
>     */Task 8 – Screen Readers
>     /*/Goal: Determine if users can easily find information on assistive technologies**/*//*
> 
>     *Scenario: *You want to find information on what a screen reader is and how it works.
> 
>     *Task: *Where would you go to find this information?
> 
>     ·*Q30: *What kind of information do you see here?**How many kinds of screen readers are there? Can you describe what a screen reader does?
> 
>     ·*Q31: *How do you feel about the amount of information? **
> 
>     ·*Q32: *On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task?**
> 
>     Task 9 – Mobile Accessibility
>     Goal: Determine if users can easily find and understand information on mobile accessibility**
> 
>     *Scenario*: You want to understand how your mobile application is doing in terms of adhering to accessibility standards.
> 
>     *Task:* Where would you go for information on best practices for mobile accessibility
> 
>     ·Q33: What kind of information do you see here?
> 
>     ·Q34: Does this information look up to date?
> 
>     ·Q35: Does this page give you the tools you need to begin building an accessible mobile application?**
> 
>     ·Q36: On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task? **
> 
>     **
> 
>     Task 10 – Carousel concepts
>     Goal: Determine if users can easily find and understand information on tutorials**
> 
>     *Scenario*: Let’s say your manager tasks you with designing an accessible carousel.
> 
>     *Task:* Where would you go if you wanted to learn more about carousels?
> 
>     //
> 
>     ·*Q37*: What kind of information do you see here?**
> 
>     ·*Q38*: Does this page have the information you need to build an accessible carousel? **
> 
>     ·*Q39*: On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task? **
> 
>     **
> 
>     Task 11 – Color Contrast
>     Goal: Determine if users can easily find and understand information on how to meet color contrast requirements **
> 
>     *Scenario*: Your manager approaches you and tells you that a page you designed did not meet color contrast requirements. How can you find information on fixing this?
> 
>     *Task:* Where would you go if you wanted to learn more about color contrast requirements?
> 
>     ·*Q40*: What kind of information do you see here? **
> 
>     ·*Q41*: Does this page help you figure out how to meet color contrast requirements?**
> 
>     ·*Q42*: What information does this page tell you about color contrast requirements for logos? **
> 
>     ·*Q43*: On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task?
> 
>     Task 12 – Identifiable Feedback
>     Goal: Determine if users can easily find and understand information on how to provide easily identifiable feedback **
> 
>     *Scenario*: You want to learn how best to let users know that there is an error on a form they submitted. (i.e. incomplete submission on a form). **
> 
>     **
> 
>     *Task:* Where would you go if you wanted to learn more about alerting the user that they submitted the wrong information?
> 
>     ·*Q**44*: What kind of information do you see here? **
> 
>     ·*Q45*: Does this give you the information you need to design accessible feedback for users?**
> 
>     ·*Q46*: On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task? **
> 
>     **
> 
>     Task 13 – Writing
>     Goal: Determine if users can easily find and understand information on how to write captions **
> 
>     *Scenario*: You write a cooking blog and you want users who are deaf and hard of hearing to be able to follow along with your demonstration videos.
> 
>     *Task:* Where would you go if you wanted to learn more about how to effectively write captions for disabled users?
> 
>     ·*Q47*: What kind of information do you see here? **
> 
>     ·*Q48*: Do you think this information gives you what you need to write captions? **
> 
>     ·**: On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task?
> 
>     */Task 14 – Presenting on WAI and WCAG
>     /*/Goal: Determine if users can easily find information from WAI and WCAG to include in a presentation. /*//*
> 
>     *Scenario:* You’re giving a presentation on accessibility at a user conference for your company and you’ve heard that WAI offers material you can use for free.
> 
>     *Task: *Where would you go to find information on using material or presentations that WAI may have already created?
> 
>     ·*Q50:*What kind of information do you see here? **
> 
>     ·*Q51: *How do you feel about the amount of information? **
> 
>     ·*Q52: *On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task?****
> 
>     **
> 
>     */Task 15 – Contact someone at wai
>     /**/Goal: Test if users can easily find information on how to contact WAI speakers /*//
> 
>     *Scenario: *You would like to contact someone from WAI to present at a conference you’re planning.
> 
>     *Task: *Where we would you look to contact someone at WAI?
> 
>     ·*Q53: *On a scale of 1-4, one being very easy and four being very difficult how would you rate the difficulty of this task? **
> 
>     Post-interface walkthrough questions
> 
>     1.Is there anything on the WAI site that stands out as being particularly annoying or frustrating?
> 
>     2.Is there anything that stands out as being particularly good?
> 
>     3.Would you recommend this web site to your colleagues? Why or why not?
> 
>     4.If this was the real world, is there any other information that you would want and which wasn’t provided or which wasn’t clear? What kind of information?
> 
>     5.What did you like most about the WAI site that we looked at?
> 
>     6.What did you like the least?
> 
>     7.If you had a magic wand and could change anything about this whole process (from start to finish), what would it be?
> 
>     Best,
>     Charlotte
> 
>     **
> 
>     Charlotte H. Wise |Lead UX Researcher & Usability Lab Manager - Visa User Experience |Phone: 512.865.2753 |Mobile:  512.632.0172 |Email: cwise@visa.com <mailto:cwise@visa.com>
> 
>     EmailSig_VBM_streamers_Medium
>