FW: WAI Web site testing: results

WSTF,
Please read Pasquale's results before the Monday teleconference. Also,
if you have questions for Pasquale, please send those asap in case he
can answer them before the teleconference.

Pasquale,

Thank you for your efforts! I added you to the WAI Site Task Force
(WSTF) mailing list (public-wai-eo-site@w3.org) for now so that we could
easily communicate about your findings.

Would you per chance be available to participate in our teleconference
on Monday (which is very late at night for you)?
WAI Site Task Force of EOWG (WSTF) Meeting Agenda for 8 November 2004
Time:	6:00pm-8:00pm US Eastern Time, for other time zones see:
		http://www.timeanddate.com/worldclock/meeting.html
Bridge:	+1.617.761.6200, 
		conference code 9783# (WSTF#)
IRC:		channel: #wstf
		server: irc.w3.org
		port: 6665

Best regards,

~ Shawn


-----Original Message-----
From: Pasquale Popolizio
[mailto:pasquale@osservatoriosullacomunicazione.com] 
Sent: Saturday, November 06, 2004 7:51 AM
To: Shawn Lawton Henry
Subject: WAI Web site testing: results


Hi Shawn,

I'm not sure I can post it to the public-wai-eo-site@w3.org list, so I
send you my test.


I asked 4 collegues and professionals to help me to test WAI Web site.
I tried also to follow AIR questionnaire.
Here are the findings.
 
Note: 2 tests were made with my help: I was near the participant; 2
tests were made by participant alone. I asked to 2 participants alone to
write and collect the path and the links they go.
After tests I asked to 4 participants their informal impressions about
wai site.
 
Tests were conducted on Nov 4 and 5.

Attachment is complete report in MS Word; I'm just preparing a XHTML
version.
 
Best regards
Ciao
 
Pasquale



---------------------------------------------
Pasquale Popolizio
IWA/HWG Member
pasquale@osservatoriosullacomunicazione.com
pasquale@popolizio.com
www.osservatoriosullacomunicazione.com
mobile 393.9402244
ICQ 267656437  AIM pasqualepoldo
----------------------------------------------







Testing WAI Web Site

http://www.w3.org/2004/09/wai-nav/8-home
 
Nov 4 and 5, 2004
 
 
 
Participants data

Number of participants:
4
 
Age:
32, 28, 40, 45
 
Occupation and title:
4 graduate: 1 web content manager, 2 marketing managers, 1 web designer
 
Role or interest as it relates to web development and/or making webpages
accessible to people with disabilities:
2 participants: to increase web professionality (web professionals);
2 participants: to increase their knowledge about web accessibility
(mktg managers)
 
In the last year, how often have they visited the Web Accessibility
Initiative (WAI) website to locate information:
3: never
1: 2 times
 
Disability:
4: none
 
How long have they been involved in Web accessibility:
2: 6 mounth
2: 1 year
 
OS and browsers used:
1 Mac OSX and Safari
2 WXP and IE6
1 W98 and IE5.5
 
 
Note: 2 tests were made with my help: I was near the participant; 2
tests were made by participant alone. I asked to 2 participants alone to
write and collect the path and the links they go.
After tests I asked to 4 participants their informal impressions about
wai web site.






Results

Task 4: Your team at work is developing a Web site and you have some
concerns about how accessible the Web site might be to people with
disabilities. Using this Web site, determine whether or not it contains
information about the basic things Web developers need to know about Web
accessibility.
 
goal: Quick Tips
* Note especially the path they take, as there are a few different ones.

 
3: yes
1: don't know
 
 
3 participants go straight to "getting started" and "introduction to
accessibility".
1 participant go instead to "guidelines and resources"
 
4 participant don't understand and don't like the "quicklinks" pulldown
menu
 
 
 
 
 
Task 5: A few of your colleagues are interested in finding out how to be
a part of WAI's effort to develop guidelines for Web accessibility.
Using this Web site, determine whether or not opportunities exist for
becoming involved in WAI Web Content guideline development.
 
* The prototype doesn't have the "answers" to this yet. However, it
would be interesting to see where they look, e.g., "WAI Groups"?
 

1 participant: "getting started" then "about wai" and "participating"
1 participant: try to click "WAI welcomes... participation from around
the world" on the right box
1 participant: don't find
1 participant: "wai groups" then "EOWG" and "how to join."
 
 
 


 
Task 6: You have just been handed a report, generated by a Web
accessibility evaluation tool, which informs you that your company Web
site contains complex information graphs that do not meet "Checkpoint
1.1." Using this Web site determine how you would meet Checkpoint 1.1
and make the graphs accessible.
 
goal: Techniques for Checkpoint 1.1
http://www.w3.org/TR/WCAG10-TECHS/#tech-text-equivalent > Long
Description of Images
http://www.w3.org/TR/WCAG10-HTML-TECHS/#long-descriptions
* This one might be too complex for now - and goes beyond WAI site
navigation to WCAG10 navigation. If you try it, note particularly the
path they take and how useful the intro pages are at helping them
understand the WCAG 1.0 documents.
 

1 participant: "getting started" then "quick tips"; then "components"
1 participant: "guidelines and resource" then "implementing"; then
"guidelines and techniques", "wcag1"
1 participant: "getting started" then "introduction to accessibility"a
nd "making a web site accessible"
1 participant: "guidelines and resource" then "developing training"
 
note: all 4 participants wish a link in home that go straight to a
section like "rules to make web site accessible"
 
 
 
 
 
Task 7: Your company is revising the online forms on its Web site. Find
specific information on how to make the online forms accessible.
 
goal: forms section in HTML Techniques
http://www.w3.org/TR/WCAG10-HTML-TECHS/#forms
* This one might be too complex for now - and goes beyond WAI site
navigation to WCAG10 navigation. If you try it, note particularly the
path they take and how useful the intro pages are at helping them
understand the WCAG 1.0 documents. Specifically, I'd like to know if
anyone sees and understands "The techniques documents are organized by
topic; for example, HTML Techniques includes sections on forms, images,
lists, links, tables, etc."

 
3 participants: don't find it
1 participant: find it
 
 
 
 
 


Task 8: You have been invited to be a presenter at a local conference on
Web accessibility. Find information on this Web site that you could use
in your talk.
 
goal: http://www.w3.org/2004/09/wai-nav/nav-train.html
* this is one I suspect will be much better than previous! :)
And a task that didn't make it into the study that would be good:
Task policies: A company with many global divisions and Web sites in
many languages has asked for your opinion on how Web accessibility laws
differ around the world. Using this Web site, determine whether or not
it contains relevant information.
goal: Home > Guidelines & Resources > General > Governmental Policies
Relating to Web Accessibility

 
 
1 participant: "getting started" then "introduction to accessibility",
then "overview to the wai slides", and then again "how people use the
web"
 
1 participant: first, try to click in the right box "support materials
to help understand and implement web accessibility"; then "getting
started" then "introduction to accessibility"
 
1 participant: "guidelines and resource" then "general resource",
"policies relating to web accessibility"
 
1 participant: "getting started" then "translation of this page", then
"overview to the wai slides"
 
 


Post-test questionnaire

For each statement, please sign a number to indicate the level to which
you agree with each statement. If you feel a statement isn't relevant to
your experiences with the Web site, select the "N/A" (not applicable)
for that item.
 
 1 = Disagree strongly
 7 = Agree strongly
 
 
The homepage is attractive 3, 5, 4, 6 average: 4.5
 
The overall site is attractive 3, 5, 5, 5 av: 4.5
 
The site's graphics are pleasing 7, 5, 4, 4 av: 5

The site has a good balance of graphics versus text 6, 7, 6, 6 av: 5.75

The colors used throughout the site are attractive 5, 7, 6, 5 av: 5.75
 
The typography is attractive 6, 7, 6, 6 av: 5.75

The homepage's content makes me want to explore the site further 1, 6,
5, 5 av: 4.25
 
It is easy to find one's way around the site 7, 7, 3, 3 av: 5
 
You can get information quickly 7, 6, 4, 4 av: 5.25

It is fun to explore the site 2, 5, 4, 4 av: 3.75

It is easy to remember where to find things 5, 7, 5, 4 av: 5.25
 
Information is layered effectively on different screens ? n/a, n/a, n/a,
n/a
 
The homepage is attention-getting 6, 7, 5, 4 av: 5.5
 
Information is easy to read 3, 7, 4, 3 av: 4.25
 
Information is written in a style that suits me 6, 5, 4, 4 av:4.75

Screens have the right amount of information 3, 7, 4, 5 av: 4.75
 
The site effectively communicates the company's image 6, 7, n/a, n/a

Information is relevant 7, 7, 7, 6 av: 6.75

The site is designed with me in mind 6, 7, 5, 4 av: 4.4
 
The site's content interests me 7, 7, 7, 6 av: 6.75
 
The site's content would keep me coming back 7, 7, 6, 7 av: 6.75
 
The site has characteristis that make it especially appealing? 3, 4, 4,
6 av: 4.25
 
The site reflects progressive, leading edge design? n/a, 5, n/a, n/a
 
The site is exciting 4, 5, 5, 5 av: 4.75
 
The site is well-suited to first-time visitors 2, 5, 3, 3 av: 3.25
 
Ths site is well-suited to repeat visitors 6, 7, 5, 6 av: 6
 
The site has a clear purpose 7, 5, 5, 5 av: 5.5

It is always clear what to do next 7, 7, 5, 4 av: 5.75
 
It is clear how screen elements work 7, 7, 5, 4 av: 5.75
 
Mistakes are easy to correct 7, 7, 3, 5 av: 5.5
 
 




Informal impressions

 
Some answers

 
A. How does the general IA and navigation design work? Is the expanding,
collapsing, end nodes, etc. intuitive?
1: not intuitive
3: intuitive
 
B. Are the "annotated navigation pages" understandable in their
relationship to the hierarchical navigation?
4: not easly
 
C. How comfortable or uncomfortable are users that when one node is
expanded the peer nodes are not (Carol's point sent in email)? 
4: quite comfortable
 
D. Is the presentation and interaction of the lower levels under UAAG
links understandable? [task: "Find the UAAG 1.0 FAQ." Then, from the
UAAG 10 FAQ, "Find the UAAG 1.0 Testimonials"]
4: a bit difficult
 
 
E. How does the category label "WAI groups" work? Does Guidelines &
Resources > Guidelines & Techniques work?
3: good; good
1: excellent; excellent
 
 
 
 
 
 
Final notes

All 4 participants wish a straight link in home to a section where it's
easy to find something like "how to make web site accessible", with tips
and techniques.
All 4 participants don't like "quicklink" pulldown menu.
All 4 participants wish a italian translation of web site; in this way
they could find sections and things more quickly. They find hard to
understand some words and concepts.