RE: thoughts points system for silver

Hi everyone,

Quick initial question: Is the context here that certain methods will require usability testing, or a more general usability testing should increase points?

Big +1s to Charles’ points about user-research:
> If attached to conformance, then it must consider the level of effort and cost associated with that practice…
> testing itself is the wrong emphasis… the act of testing is not an indicator that … those changes had any measurable human impact.

And Rachael’s point about process:
> I've noticed there are two approaches. One.. evaluates results. The other … measures process that should ensure results.  … But usually we test the results which in UX is number of clicks, time to complete, number of errors, etc depending on the usability measure being tested.

I’m assuming those measures for things like number of clicks, time taken, errors are relative to the same site before changes? I.e. a before/after.

Whilst I am very much in favor of user-research methods being a better ‘tool for the job’ than guidelines for certain aspects of accessibility, I think it would be futile to try and award points based on the results or even impact of the testing.

It would not apply across different types of sites (let alone size of organizations), which is the point of guidelines. Comparing the task-completion time for an email app and a News website wouldn’t make sense.

Also, as (Charles I think?) pointed out, if you are iterating on a website or design pattern, you don’t need the same level of (UX) testing as you did originally. Having a one-size testing regime would in some cases incur cost for little benefit.

How about working backwards from what we know produces better results?

For example:

  *   If an organization does not conduct user-research, the first step is to do so.
  *   If an organization does some user-research, include People with Disabilities (PwD).
  *   If an organization does include PwD in end-of-project usability testing, increase the inclusion further up the process.

In terms of increasing points: I would be more in favor of the ‘measures process that should ensure results’ approach, unless they are quite specific to a method.

That also helps with the prevention rather than remediation aspect, and helps to provide a roadmap of improvement over time.

Cheers,

-Alastair