Re: Agenda item for today's meeting - review test questions

Hi Charlotte,

Just FYI:

We won’t have a working search for the testing (as the prototype 
won’t be publicly indexable by search engines and our own custom 
search is not ready).

Eric

On 20 Sep 2017, at 18:43, Wise, Charlotte wrote:

> Hi all,
>
> Here are the general questions we plan to ask during the usability 
> testing the first week of October. I'd like to discuss during this 
> week's call as the test plan will be finalized and tested internally 
> next Thursday.
>
>
> OBJECTIVES
>
> ·         How do people interact with the WAI web site? Does the home 
> page convey the breadth and scope of the resources WAI provides?
>
> ·         Is the information architecture for the site clear, 
> navigable, and does it assist users from different audiences in 
> finding the content they seek?
>
> ·         Is the visual design and page layout aesthetically pleasing 
> to visitors?
>
> ·         Does the content provided on the site make sense, meet the 
> needs of different audiences, and establish WAI as a credible 
> accessibility resource?
>
> Intro Questions
>
> 1.       Tell me a little bit about your current job. Where do you 
> work and what is your role?
>
> 2.       Can you describe for me what accessibility is in your own 
> words?
>
> 3.       Do you have any experience with accessibility?
>
> 4.       Can you describe how accessibility is relevant to your work, 
> or how it might impact your work in your current position?
>
> 5.       What resources do you use to get answers when you have 
> accessibility-related questions?
>
> o   What makes this resource useful?
>
> o   What do you get out of it?
>
> 6.       Let's say you needed to solve an accessibility problem for a 
> project. Walk me through how you would go about finding that 
> information.
>
>
> Task 1 - First Impressions
> Goal: Gauge how effective the homepage is in communicating information 
> to users
>
> Task:  Take a look at the homepage of the WAI website.
>
> ·         Q1: Have you visited this site before?
>
> o   Where do you typically go to when you visit this site?
>
> ·         Q2: What is your first impression here?
>
> o   What do you think you can do on this site?
>
> ·         Q3: Before we begin our tasks, let's take a look at the 
> navigation briefly. What kinds of information do you think the 
> different section headings would contain?  [Walk participants through 
> each section and probe on specific questions noted]
>
> o   Where do you think you would go to find Tutorials? (It's under 
> Design & Develop)
>
> o   Accessibility Fundamentals - probe on
>
> §  What do you think "standards harmonization" mean?
>
> §  What do you think "diverse abilities and barriers" means in 
> relation to people with disabilities? (nested under How People With 
> Disabilities Use The Web)
>
> o   Plan & Manage - probe on
>
> §  What do you think "Approaches for Interim Repairs?" mean?
>
> o   Test & Evaluate - what does "Using Combined Expertise" mean to 
> you?
>
> ·         Q4: How can you tell if a section in the secondary 
> navigation has additional content?
>
> ·         Q5: Which sections interest you the most or would be the 
> most relevant to your work?
>
> Task 2 - Get Started with Accessibility
> Goal: Determine if users can easily find and understand general 
> information on accessibility
> Scenario: You're working on a website for work and you've been told 
> you need to make it accessible, but you don't know where to start or 
> what that really means.
>
> Task: Where would you go to find information on getting started on 
> accessibility?
>
>
> ·         Q6: What kind of information do you see here? How do you 
> feel about this overview?
>
> ·         Q7: How do you feel about the amount of information on this 
> page?
>
> ·         Q8: After reading this, what do you need to do get started 
> with web accessibility?
>
> ·         Q9: How would you define what accessibility is in your own 
> words?
>
> ·
> Task 3 - Building a Business Case
> Goal: Determine if users can easily find information supporting the 
> business case for accessibility
> Scenario: You're having trouble convincing your boss to make 
> accessibility a priority for an app that your team is building and are 
> wondering if WAI has any information that could help you make the case 
> for him.
>
> Task: Where should you go to find information to help you convince 
> your boss?
>
>
>
> ·         Q10: What kind of information do you see here?
>
> ·         Q11: How do you feel about the amount of information?
>
> ·         Q12: From this page, where would you go to find financial 
> arguments for prioritizing accessibility? (Path is WAI Home --> 
> Getting Started --> Business Case --> Financial Factors)
>
> ·         Q13: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
> Task 4 - Legal information
> Goal: Determine if users can easily find legal information related to 
> accessibility
> Scenario: Your company is starting to do a lot more work with a 
> partner firm in Mexico. You want to know what the laws are in Mexico 
> concerning accessibility so you can determine how that might affect 
> your company.
>
> Task: Where would you go to find this information?
>
>
>
> ·         Q14: What kind of information do you see here?
>
> ·         Q15: How accurate do you think this information is?
>
> ·         Q16: How relevant do you think this information is?
>
> ·         Q17: How do you feel about the amount of information?
>
> ·         Q18: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
>
> Task 5 -  Developing an Accessibility Policy
> Goal: Determine if users can easily find information on developing an 
> accessibility policy for their organization.
> Scenario: Your company has recently begun considering their approach 
> to accessibility. Your manager has asked you to do some research on 
> developing an accessibility policy for your organization.
>
> Task: Where we would you look to find this information?
>
>
> ·         Q19: What kind of information do you see here?
>
> ·         Q20: How helpful do you think you would find this page in 
> writing an accessibility policy?
>
> ·         Q21: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
>
> Task 6 - Easy Checks
> Goal: Determine if users can easily find and understand how to begin 
> testing for accessibility
> Scenario: Let's say you want information on how to get started 
> assessing the accessibility of a web page.
>
> Task: Where would you go to find information on get started assessing 
> the accessibility of a web page?
>
>
>
> ·         Q22: What do you think the term "Easy Checks" means?
>
> ·         Q23: What kind of information do you see here?
>
> ·         Q24: Based on what you see on this page, what are Easy 
> Checks?
>
> ·         Q25: How useful is this information in helping you get 
> started on assessing accessibility?
>
> ·         Q26: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
>
>
> Task 7 -Users with Specific disabilities
> Goal: Determine if users can easily find and understand information on 
> people with specific disabilities
>
> Scenario: You are interested in learning more about how users with 
> deafness or hearing disabilities experience the web.
> Task: Where would you go to find information on how users with hearing 
> disabilities experience the web?
>
>
>
> ·         Q27: What kind of information do you see here? How do you 
> feel about this overview?
>
> ·         Q28: How do you feel about the amount of information?
>
> ·         Q29: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
> Task 8 - Screen Readers
> Goal: Determine if users can easily find information on assistive 
> technologies
> Scenario: You want to find information on what a screen reader is and 
> how it works.
>
> Task: Where would you go to find this information?
>
>
>
> ·         Q30: What kind of information do you see here? How many 
> kinds of screen readers are there? Can you describe what a screen 
> reader does?
>
> ·         Q31: How do you feel about the amount of information?
>
> ·         Q32: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
> Task 9 - Mobile Accessibility
> Goal: Determine if users can easily find and understand information on 
> mobile accessibility
> Scenario: You want to understand how your mobile application is doing 
> in terms of adhering to accessibility standards.
>
> Task: Where would you go for information on best practices for mobile 
> accessibility
>
>
> ·         Q33: What kind of information do you see here?
>
> ·         Q34: Does this information look up to date?
>
> ·         Q35: Does this page give you the tools you need to begin 
> building an accessible mobile application?
>
> ·         Q36: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
>
>
> Task 10 - Carousel concepts
> Goal: Determine if users can easily find and understand information on 
> tutorials
> Scenario: Let's say your manager tasks you with designing an 
> accessible carousel.
>
> Task: Where would you go if you wanted to learn more about carousels?
>
>
> ·         Q37: What kind of information do you see here?
>
> ·         Q38: Does this page have the information you need to build 
> an accessible carousel?
>
> ·         Q39: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
>
> Task 11 - Color Contrast
> Goal: Determine if users can easily find and understand information on 
> how to meet color contrast requirements
> Scenario: Your manager approaches you and tells you that a page you 
> designed did not meet color contrast requirements. How can you find 
> information on fixing this?
>
> Task: Where would you go if you wanted to learn more about color 
> contrast requirements?
>
>
>
> ·         Q40: What kind of information do you see here?
>
> ·         Q41: Does this page help you figure out how to meet color 
> contrast requirements?
>
> ·         Q42: What information does this page tell you about color 
> contrast requirements for logos?
>
> ·         Q43: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
>
>
> Task 12 - Identifiable Feedback
> Goal: Determine if users can easily find and understand information on 
> how to provide easily identifiable feedback
> Scenario: You want to learn how best to let users know that there is 
> an error on a form they submitted. (i.e. incomplete submission on a 
> form).
>
> Task: Where would you go if you wanted to learn more about alerting 
> the user that they submitted the wrong information?
>
>
> ·         Q44: What kind of information do you see here?
>
> ·         Q45: Does this give you the information you need to design 
> accessible feedback for users?
>
> ·         Q46: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
>
> Task 13 - Writing
> Goal: Determine if users can easily find and understand information on 
> how to write captions
> Scenario: You write a cooking blog and you want users who are deaf and 
> hard of hearing to be able to follow along with your demonstration 
> videos.
>
> Task: Where would you go if you wanted to learn more about how to 
> effectively write captions for disabled users?
>
>
>
> ·         Q47: What kind of information do you see here?
>
> ·         Q48: Do you think this information gives you what you need 
> to write captions?
>
> ·          : On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
>
> Task 14 - Presenting on WAI and WCAG
> Goal: Determine if users can easily find information from WAI and WCAG 
> to include in a presentation.
> Scenario: You're giving a presentation on accessibility at a user 
> conference for your company and you've heard that WAI offers material 
> you can use for free.
>
> Task: Where would you go to find information on using material or 
> presentations that WAI may have already created?
>
>
> ·         Q50: What kind of information do you see here?
>
> ·         Q51: How do you feel about the amount of information?
>
> ·         Q52: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
>
> Task 15 - Contact someone at wai
> Goal: Test if users can easily find information on how to contact WAI 
> speakers
> Scenario: You would like to contact someone from WAI to present at a 
> conference you're planning.
>
> Task: Where we would you look to contact someone at WAI?
>
>
>
> ·         Q53: On a scale of 1-4, one being very easy and four being 
> very difficult how would you rate the difficulty of this task?
>
>
>
> Post-interface walkthrough questions
>
>
>
>
> 1.       Is there anything on the WAI site that stands out as being 
> particularly annoying or frustrating?
>
>
>
> 2.       Is there anything that stands out as being particularly good?
>
>
> 3.       Would you recommend this web site to your colleagues? Why or 
> why not?
>
>
> 4.       If this was the real world, is there any other information 
> that you would want and which wasn't provided or which wasn't clear? 
> What kind of information?
>
>
>
> 5.       What did you like most about the WAI site that we looked at?
>
>
> 6.       What did you like the least?
>
>
> 7.       If you had a magic wand and could change anything about this 
> whole process (from start to finish), what would it be?
>
> Best,
> Charlotte
>
>
>
> Charlotte H. Wise | Lead UX Researcher & Usability Lab Manager - Visa 
> User Experience | [Phone:]  512.865.2753 |   512.632.0172 | [Email:]  
> cwise@visa.com<mailto:cwise@visa.com>
>
> [EmailSig_VBM_streamers_Medium]





--

Eric Eggert
Web Accessibility Specialist
Web Accessibility Initiative (WAI) at World Wide Web Consortium (W3C)