Re: MobileOK Tests (was [agenda] Agenda for BPWG Call 2008-02-07)

On Feb 8, 2008 7:29 AM, Alan Chuter <achuter@technosite.es> wrote:
> * Checker-friendly
> * Human testable
> * Not testable
>
> This may seem obvious but it didn't seem to be so the other day. The
> first two together can still produce a useful label, while the third
> type are useful even if they can't be tested. what is unfortunate is
> that there has been an implicit prioritising of the BPs on the basis
> of their suitability for the checker which it's perhaps time to
> examine again before doing the same on the basis of human testability.

You're absolutely right, some will never be testable, and that's OK. I
think the appendix at the end of mobileOK Basic lists this out too in
a different way.

Yes, we put everything that could be machine-testable into Basic,
because it is machine-testable, and I also think that's entirely fine.
You could also try to prioritize BPs from most important to least
important, and pick some of the most important ones for Basic, whether
or not they were machine-testable. That seems like the first, most
natural approach.

But then we would have ended up with two test suites that aren't of
much use to all but a handful of people on the planet, since they
would need a certified mobileOK evaluator to test. So, Basic could
have been the important tests that are also machine-testable, since
there is such an overwhelming practical utility to providing an
automated tool. But then there were complaints that it was "too easy."
So, everything that's possibly machine-testable was put in.

mobileOK Pro would just add everything else that can be tested about
the BPs. I think this ends up being a very practical and sensible
approach.

Received on Friday, 8 February 2008 16:26:42 UTC