Re: MobileOK Tests (was [agenda] Agenda for BPWG Call 2008-02-07)

On Feb 8, 2008 11:42 AM, Paul Walsh <paul@segala.com> wrote:
> I agree. It was originally agreed that the mobileOK basic would be
> made up of tests which were relatively easy to comply with. Whether
> these were machine-testable or not, was not relevant. However,
> further down the line, this changed for some very strange reasons. We
> now have mobileOK basic which is machine-testable - even if they
> aren't necessarily easy to comply with.

Again the logic was: it's overwhelming useful to provide something
machine-testable. So let's put into Basic everything that is easy to
comply with or important, and also machine testable. But then there
were repeated claims that the suite was too easy -- this is the reason
you are looking for. So it became everything machine testable. I think
that's fine. It's a simple condition, and has resulted in a useful
specification and tool.

> "since they would need a certified mobileOK evaluator to test"
>
> What do you mean by this?

Well let's say we put out a suite of tests. Does X pass it or not? if
it's machine testable, then you run some presumably correct
implementation of the machine tests on X. You could implement it
yourself but you don't have the time, and are anyways not sure you
will come out with the "right" answers. If anyone then cares to ask
why you pass, it's not so great to say, well, I wrote code that says I
pass. It would be much more credible to say that a third-party system
passed you.

Analogously, for a human testable suite, you might suggest that
individual developers are going to effectively implement mobileOK Pro
themselves by becoming an expert on mobileOK Pro, executing the tests
and assuming they're right, and then feel it's useful to say, I pass
the tests because I say so. I assume this model really also entails
some third-party "implementation", and that involves humans, and
involves time and money.

>From there we get into the same old circles... there is no doubt some
market for this, but at the moment, given that mobileOK Basic is not
exactly taking the world by storm, I think we do need to honestly ask
who mobileOK Pro is for. Is there at least one potential customer
someone can identify? I do hear strong convictions here that this is
not only important to do, but that it's somehow dangerous not to do
it. I would just like to continue to understand why.

Received on Friday, 8 February 2008 17:13:19 UTC