RE: Some draft code for mobileOK Basic Tests RI

> -----Original Message-----
> From: Sean Owen [mailto:srowen@google.com]
> Sent: 06 February 2007 20:42
> To: Jo Rabin
> Cc: public-mobileok-checker@w3.org
> Subject: Re: Some draft code for mobileOK Basic Tests RI
> 
> On 2/6/07, Jo Rabin <jrabin@mtld.mobi> wrote:
> > > But the hard part, the heavy lifting is of course producing that
> > > intermediate doc. Implementing code that produces this
intermediate
> > > representation seems like it can't be done in a language-neutral
way.
> > > There are some details here like wrestling with encoding and image
> > > formats and CSS that can't be described by a transform.
> > >
> > > Given that, I don't yet see the value in the additional code and
> > > performance overhead needed to generate the intermediate document.
It
> > > may save a little bit of additional code, but, it's the code for
the
> > > easy part, like running some XPath queries.
> > >
> > Well, I'm not clear what additional code is involved? You need to do
all
> > the (pre)processing anyway, so why not just serialise the results of
> > that processing as XML. That makes the information available not
just to
> > the mobileOK Basic tests but also to some aspects of mobileOK (full)
and
> > to other ad hoc tasks.
> 
> Are we talking about the results of the tests? it seems obviously
> useful to make test results available in some XML format.

Yes we are agreed on that. But in addition, we are talking about
normalising the format of the input to the tests, i.e. creating a meta
document. 

> 
> I believe the pre-processing is the hard part and must already be
> implemented in a concrete language like Java. That seemed to make the
> idea of a platform-neutral implementation infeasible, so this doesn't
> obviously help the goal of portability.

Well I'd argue that by separating the problem into normalising the
information input, and having a single representation of the tests
themselves, the problem of worrying whether two different-language
instances are the same is reduced to worrying about whether any instance
normalises correctly.

> 
> Extensibility seems like the key point here -- exposing more about
> what happens during the tests. I can imagine recording so much
> information about the document and results of accessing it and linked
> resources and errors and so on, such that so much processing has been
> done already that an XSLT's simple conditional logic is enough to
> implement the last 5% of the test logic and produce results.

Well 5% seems a bit harsh to me. I agree that XSLT is a bit clunky but
... the test that would give me sleepless nights, if any, is SCROLLING.
Mind you I am a bit dubious about the complexity/value trade-off in that
test anyway. The best it can do is WARN.

> 
> In particular, I think that "if" statements in the tests present a
> problem. I imagine we have to collect information about both the "if"
> and "else" branches of the test so that the XSLT can select between
> the two later... or else put that logic into the preprocessing, which
> detracts from the point of the XSLT to begin with.

Well, I knocked together some XSLT by way of example. Attached to this
post. example1.xml is a hypothetical input file and example.xslt is an
xslt that processes it and handles auto_refresh. [incidentally, some
thought needs to be given as to whether to check against the requested
URI or the final URI following any 3xx redirections to get there] 

> 
> In and of itself, it does not seem worth the effort. But again,
> extensibility may indeed be worth the trouble. I know you guys are
> going to want to add on to the tests and this must be accommodated.
> 
Yes, we are, explicitly per James's post on this thread.

> What about greatly expanding the information supplied with the test
> results? If I'm right, and preprocessing is 95% of the work, then
> producing a rich intermediate state doc is not so different from
> producing a rich results document. For example, James's idea to supply
> the repsonse status, headers, body, etc in the results seems entirely
> reasoanble -- and surely that can be serialized to XML. If that's what
> we're talking about then we entirely agree.
> 
> I suppose I'd also just imagined that anyone who wanted to augment the
> tests would probably have to change the code to begin with, and it's
> open source, so, no big deal.
> 
> If we're talking about extensibility in the context of localization,
> then that can surely be handled directly.

Well on this point, Dom suggested that the output might include codes to
aid localization, and though it is (somewhat) off-topic, I wonder if
that is needed then should these codes be normative (i.e. be defined in
the mobileOK document).

> 
> What kind of extensibility does this approach meaningfully buy you,
> or, what's the use case you have in mind?

I think the attraction of it is per James's post.

I think the point of divergence will be whether you are an XSLT wonk or
a Java wonk, and depending on your viewpoint you either 'simply' hack a
bit of Java to extend, customise, produce a different version of the
output/report, or you 'simply' hack a style-sheet.

> 
> 
> > > I think any kind of prototype is great at this point as we knock
> > > around ideas. Prototyping the bit that produces this intermediate
XML
> > > document seems like the most crucial next step for that approach.
We
> > > probably want to define an XML format for the test results too.
> >
> > Agreed with that, but I guess as part of a decision making process.
We
> > do need to come to a conclusion in a reasonably timely way. So, what
do
> > we need to know and in how much detail do we need to know it?
> >
> > Just to repeat dotMobi's invitation to discuss all this F2F in
Dublin.
> > If we could assess willingness to attend, timescale and agenda, then
I
> > think that would frame prototyping and other preparatory activities
...
> 
> Agreed and roughly speaking we need to figure out how to proceed in
> the next two weeks so as to have something reasonable to show by the
> release of mobileOK Basic Tests.
> 
> I just saw James's additional email and I think we mostly agree that
> the tests should generate a lot of information beyond simple results,
> and in XML form, to aid extensibility. I agree. I suppose I only ask
> why it has to be something different from the results doc.

As I see it the input doc and result doc are rather similar. You can
imagine that the output doc would optionally include parts of the input
doc by way of an audit trail, as James says.

> 
>  I am not sure I can make it to Dublin by next week, so how about a
> conference call for all interested parties? If we're really fairly
> close in opinion I think we can sort that out in a call. Feel free to
> pick a time that's early for the east coast of the US.
> 
I'd be up for that, but am mindful of the impending 3GSM which is
consuming most people's attention at the moment. Next week is 3GSM,
following week is post-3GSM triste, at a guess (and I am on vacation). 

Let's start a separate thread to fix a date for a call (talk about
meta-activity!).

Jo