RE: MobileOK Tests (was [agenda] Agenda for BPWG Call 2008-02-07)

I'm fine with some tests being excluded as "non testable", however think
we need to be clear that there are more ways of testing than just the
visual assessment of the output from a site. 

For example, TESTING is in principle verifiable by doing a simple audit
of the development process and verifying that test records show that the
process was followed. I'm personally not in favour of specifying this
audit type of approach.

Jo



> -----Original Message-----
> From: public-bpwg-request@w3.org [mailto:public-bpwg-request@w3.org]
On
> Behalf Of Alan Chuter
> Sent: 08 February 2008 12:29
> To: MWI BPWG Public
> Subject: MobileOK Tests (was [agenda] Agenda for BPWG Call 2008-02-07)
> 
> 
> During and following the F2F the other day I have been reflecting on
> what we are trying to do and whether it is worthwhile and achievable
> (as discussed the first day with Jo). Supposedly the basic tests are
> the most important, although apart from saying "basic steps" should
> have been taken, the document seems to dodge discussing the basis for
> selecting the BPs to test. Perhaps I'm mistaken but it seems to me
> that what was selected was a set of tests to suit the checker not the
> needs of the user.
> 
> Now with Pro we are saying that Basic is not sufficient and that a
> human tester is needed to test for what will give an adequate
> experience. This is reasonable, but even with a manual operator I
> don't believe that all the BPs can be tested. For example:
> 
> * TESTING (Determine whether the content was tested with a real device
> not an emulator)
> * LIMITED (Determine what the user requested; What user? When?)
> * ERROR_MESSAGES (Discover all error messages a site can produce)
> 
> These are not ambiguous or unquantifiable, but simply unknowable. I
> think we need to accept that some BPs can not be tested, but not be
> put off by this but simply exclude them. We then have three classes of
> BP:
> 
> * Checker-friendly
> * Human testable
> * Not testable
> 
> This may seem obvious but it didn't seem to be so the other day. The
> first two together can still produce a useful label, while the third
> type are useful even if they can't be tested. what is unfortunate is
> that there has been an implicit prioritising of the BPs on the basis
> of their suitability for the checker which it's perhaps time to
> examine again before doing the same on the basis of human testability.
> 
> best regards,
> 
> Alan
> 
> 
> 
> 
> 
> On 07/02/2008, Scheppe, Kai-Dietrich <k.scheppe@telekom.de> wrote:
> >
> > Summary of mobileOK Pro F2F in London:
> >
> > Without much ado, here is a summary (from memory, since I don't have
any
> > minutes at the moment) what was done and agreed upon....
> >
> > Item: Concerns from Jo, Sean, Dom regarding the existance of the
group
> > Jo was kind enough to join us on Wednesday and we addressed the
various
> > points with him in the meeting.
> > 1) legitimization of the TF
> > Discussion about this was tabled as views changed.  Once there was
great
> > support, now there is less.  However, it remains for the TF to
> > demonstrate that it will do the work in time, which seems to be the
main
> > concern
> > 2) implementation experience of mobileOK Basic needed for mobileOK
Pro
> > This was accepted as a possibility, but the TF strongly suggests
that in
> > the absence of existing mobileOK Pro all discussion about this point
is
> > moot.  If there is no interest in the public in general, then
mobileOK
> > Pro may whither away, but at the moment neither positive nor
negative
> > claims can be made.
> > 3) lack of consumer pull for mobileOK Pro
> > see 2)
> > 4) potential lack of meaning of self-certified compliance to
mobileOK
> > Pro
> > A claim will remain a claim, until certified.  This has been the
nature
> > of mobileOK and will remain so.
> > Furthermore, the educational approach of mobileOK still exists where
> > BPWG hopes to enlighten the public that there are better ways to
build
> > mobile friendly sites.  mobileOK Pro is essential in that.
> > Lastly, the tests are designed to offer bracketing, when needed, to
aid
> > in coming to the right test results.  As was point out by Sean many
> > times, just having people adhere to mobileOK Basic would be a great
> > step.
> > The same goes for mobileOK Pro.  If people even think about and
consider
> > their actions in light of mobileOK Pro, much will be helped.
> > 5) points which have been dropped from WCAG and which may have
> > similarities with mobileOK Pro.
> > Alan will come back to the group with information where this may be
> > relevant
> >
> >
> > Item: Milestones
> > By Feb 21:
> > - submission more detailed charter
> > - submission 1st draft of mobileOK Pro Tests
> > - request for issue of 1st public working draft of mobileOK Pro
Tests
> >
> > Next F2F March 19
> > - reworking with feedback from group and public
> >
> > June
> > - LC in June
> > - CR End of June
> >
> >
> > Item: mobileOK Pro Tests
> > - We went through all tests, save one which we simply didn't get to.
> > - the document will be edited on Googledocs and is already being
> > modified as we speak (task is to reexamine the tests and flesh out
> > details)
> > - TODO: Reexamination of the mobileOK Basic Tests for partial
coverage
> > of BPs, followed by creation of additional Pro Tests if needed
> >
> >
> > Item: Notes to BPWG and Open issues
> > - We have identified several points we wish to communicate to the WG
> > about some of the BP, especially for consideration BP 2.0
> >
> >
> >
> >
> > Kai
> >
> >
> >
> 
> 
> --
> Alan Chuter,
> Senior Web Accessibility Consultant, Technosite (www.technosite.es)
> Researcher, Inredis Project (www.inredis.es/)
> Email: achuter@technosite.es
> Alternative email: achuter.technosite@yahoo.com
> Blogs: www.blogger.com/profile/09119760634682340619

Received on Monday, 11 February 2008 11:40:11 UTC