Re: Usability Testing first of April [was: Fwd: v2 -- Re: Tasks - Resource mapping]

Thank you, Charlotte, for this information. I agree with your perspectives on optimizing user input and usability testing.

Originally I think y'all said you had time the last week of March. How about we do some virtual f2f work -- that is, set aside a couple-few hours to do remote worksession(s) together. We can use WebEx video or other if that would be helpful.

Best,
~Shawn

On 3/17/2017 3:05 PM, Wise, Charlotte wrote:
> Hi Shawn,
>
> I agree that user testing is essential and am glad that everyone supports it. I recommend that we start first with another Treejack test when we have a third draft of the IA, followed up by a short task-based test of that IA situated within the context of Alicia's mockups similar to what you describe below (I would recommend using a remote testing service like usertesting.com for this), and finishing with an in-lab based summative test of the final draft IA and design along with several pages of the revamped content.
>
> It is pretty expensive to go straight to in-lab testing (recruitment per participant is roughly $3-400/pp for ~10 participants, plus travel costs for everyone to get here) and I don't think we are in a state yet with the site where an in-lab test will elicit more useful, nuanced feedback than the less-expensive options we can do first.
>
> Also, we know that the second-draft IA that I Treejack tested with does not work very well as it is. It needs *substantive* doc title changes (probably not minimal ones) and possibly structural changes, along with (I suspect) corollary substantive rewrite or reorganization of the content. I don't think that re-testing the draft we currently have is a good use of our collective resources.
>
> I like some of the ideas you suggested in the IA you sent earlier this week and will incorporate some of them as I work on finishing a third draft. I don't think that a primary navigation structure with just four categories adequately addresses the complexity of the content we have, though I like simplicity too, and that’s definitely my primary goal.
>
> When we get to the point where we want to start testing Alicia's mockups, in order to validate that the design and IA meets the needs of finding top task information, I think we would need to first build out most of the site or somehow port over all the content as there will need to be content on each page within the mockups in order for users to have a realistic experience. (This is why a tool like Treejack is an easy first step in validating IA--there's no need for content build out.) I am guessing that will take some time, as there is not currently a 1:1 migration path from the current sitemap to the new IA. Once we are ready to do in-lab testing, there will be a roughly 6-8 week lead time between writing the test plan, reviewing and revising it, recruiting participants, and then sitting down to the sessions. I will treat our this project  just like one of our Visa clients, and follow the same process.
>
> Just so we are all on the same page--we are presently considering 6 primary navigation categories: About WAI, Understand, Promote, Evaluate, Implement, Participate. I've asked Alicia to consider these in the context of the new design, and also to begin thinking about placing a link to the guidelines themselves in the utility navigation or elsewhere that they would be omnipresent. We know that users weren't able to easily find them when nested in the primary navigation from the initial Treejack test I did.
>
> I think our next step is for me to take a third pass through the IA using the content that you and Sharon mapped, review it together, and then Treejack test it. If it passes a Treejack test, we can consider timing for user testing. We can then look at what effort would be required to build out a somewhat or mostly-functional prototype site usable for testing based on the third draft IA.
>
> I would like to wait on setting a date for us to convene in Austin until James is back and we can discuss what would be the best timeline for that. Currently, we have studies booked in our lab for every week in April (I have some leeway around participating in every one of them, but not a ton). I don't recall what our initial idea was behind why we were wanting to get together in person (I have a toddler who isn't sleeping well and it's been a few weeks since we discussed!) but I'd like to re-establish that agenda as well. (James is OOO all of next week traveling to California for business.)
>
> Best,
> Charlotte
>
> -----Original Message-----
> From: Shawn Henry [mailto:shawn@w3.org]
> Sent: Wednesday, March 15, 2017 11:27 AM
> To: Sharron Rush <srush@knowbility.org>; Wise, Charlotte <cwise@visa.com>; Green, James <jgreen@visa.com>; Watson, Caleb <calwatso@visa.com>
> Cc: WSTF <public-wai-eo-site@w3.org>
> Subject: Usability Testing first of April [was: Fwd: v2 -- Re: Tasks - Resource mapping]
>
> On 3/7/2017 3:08 PM, Sharron Rush wrote:
>> - More user testing Shawn liked this idea. In fact when we spoke
>> today, we wondered if we used the current design along with a version
>> of the top level IA we evolved from the card sort - Implement,
>> Promote, Understand, Participate. We can arrange resources within
>> these categories and see how people can find them.
>
> Summary:
> I would like to propose that we do usability testing asap with a rough prototype of the latest visual design from Alicia; nav under main categories tweaked from Charlotte's draft IA and the card sort: Understand, Implement, Evaluate, Promote, (with other sections in the "utility area" and footer, per Alicia's design); and some very minimal doc title tweaks and landing page tweaks.
>
> Background:
> * Lots of us suggest usability testing early. (ftr, I didn't think it would have been effective to do it on the current outdated website, especially given limited resources.) I think we have enough ideas now to get valuable input from UT.
> * Charlotte reported on IA testing (via TreeJack) that users found it very difficult and they had a high failure rate with the draft IA and resource renaming from <https://lists.w3.org/Archives/Public/public-wai-eo-site/2017Feb/0007.html> (minor update from <https://lists.w3.org/Archives/Public/public-wai-eo-site/2017Feb/0001.html> with guidelines under Understanding Accessibility, afaik).
> [side note: If there's not a separate category for Guidelines, I think they fit mostly under Implement, but is also pointed to from Understanding and Evaluate.]
>
> Setting dates:
> I can travel to Austin for UT and followup -- if we set dates so I can get decent flight costs and times. It would work best with my schedule to be in Austin Wed 5 and Thur 6 April (and possibly Tuesday afternoon).
>
> Thoughts?
>
> Best,
> ~Shawn
>
>

Received on Tuesday, 21 March 2017 16:19:16 UTC