RE: Some draft code for mobileOK Basic Tests RI

Declarative test definitions is clearly highly desirable and although I
am concerned about performance, I guess that most implementations would
use XPath internally anyway, so there is probably not much lost.

Ref loading invalid markup into a DOM, I've done a little playing around
with this in PHP, which seems pretty forgiving (e.g. doesn't mind WML
all that much). However, I wonder if depending on the vagaries of the
language specific tools is good enough? Say the chosen tools in Java and
php were equally flexible, but had different approaches, would this
matter? 

I think it probably does, as the tests need to be run against a specific
input - in any case, if it does matter, would it be sufficient (for the
purposes of getting a DOM from an invalid document) to use some
specified version of Tidy?

The test results would need to include both the invalid input and the
tidied input.

Jo

> -----Original Message-----
> From: public-mobileok-checker-request@w3.org [mailto:public-mobileok-
> checker-request@w3.org] On Behalf Of Sean Owen
> Sent: 05 February 2007 16:23
> To: James G Pearce
> Cc: public-mobileok-checker@w3.org
> Subject: Re: Some draft code for mobileOK Basic Tests RI
> 
> 
> I do agree that the tests should take advantage of all relevant
> technologies and libraries. For example using XPath makes sense for
> many tests, and XPath is implemented in Java, and writing tests in
> terms of such queries makes it easier to port to another language
> where some library implements XPath queries too.
> 
> You raise another good point, that unless we can get a DOM out of the
> document, most tests like this can't run. And lots of docs won't be
> well-formed. One alternative is to write a fallback tag-soup
> implementation of the tests to try to proceed even when the doc is not
> well-formed, but, that seems wasteful. I like the idea of being
> lenient or trying hard to get a DOM even when the doc isn't
> well-formed. I haven't played with this too much, but, I hope that
> standard XML parsers can do this to some extent.
> 
> So I think the code here does a similar thing in the form of the
> TestContext object -- contains the parsed DOM (Document) if any,
> information about the resource that was retrieved (HTTP headers, raw
> body), and other common state. All tests are implemented in terms of
> this input. Is that what you had in mind?
> 
> Test output -- for the pure library implementation, I'd like to return
> results as a series of objects rather than as XML, so that callers
> aren't forced to gratuitously re-parse the XML. But, no reason it
> can't get very easy to render that object tree into XML by the library
> -- some kind of TestResults.toXML() method. There are certainly
> libraries that can do this automatically.
> 
> On 2/5/07, James G Pearce <jpearce@mtld.mobi> wrote:
> >
> >
> >
> > Hi Sean,
> >
> > Nice framework! Thanks.
> >
> > One idea we had was to try to aim to describe as many of the tests
as
> > possible in a language-agnostic way. The test descriptions could be
> picked
> > up at run time. This would have a number of advantages:
> >
> >  * Easier porting of the engine (it's just a test interpreter)
> >
> >  * Decoupling the implementation from any changes made to the tests
> > themselves
> >
> >  * Makes it easy for 3rd parties to add additional tests (at least
those
> > which can be described in that way).
> >
> >
> > As for the way to describe the tests, there are probably plenty of
> common
> > approaches. But it basically would need to be a query that is
applied to
> a
> > possibly-XML document (and its headers, and its dependencies) and
which
> > returns a tri-state: passed/failed/not-run.
> >
> > One approach I've used in the past for this sort of thing is Xpath
(The
> > "Page must not contain frames" test is codified as ".//frameset
returns
> 0
> > nodes"), but we'd need to pre-process the document against which the
> Xpath
> > is run to make sure it a) was coaxed into being well-formed, b)
> contained
> > meta-data like HTTP headers and binary object sizes etc, as required
to
> > evaluate the tests.
> >
> > e.g.
> >
> >  // ns1 = fetched HTML
> >  // ns2 = pre-processed additions
> > <ns2:document>
> >  <request method='get'>
> >   <header name='...' value='...' />
> >   <header name='...' value='...' />
> >   <body />
> >  </request>
> >  <response status='200''>
> >   <header name='...' value='...' />
> >   <header name='...' value='...' />
> >   <body size='4523' totalsize='23325' xmlvalid='true'
> xmlwellformed='true'>
> >    <ns1:html>
> >     <head>...</head>
> >     <body>
> >      <p>
> >       <img src='aa.gif' ns2:status='200' ns2:size='1575'
> > ns2:content-type='image/jpg' ns2:width='200px' ns2:height='150px'
> > ns2:depth='256'  />
> >
> >      </p>
> >      <p>
> >       <a href='other.html' ns2:status='200' ns2:size='18234'
> > ns2:content-type='text/html' />
> >      </p>
> >     </body>
> >    </ns1:html>
> >   </body>
> >  </response>
> > </ns2:document>
> >
> > A nice side-effect of this sort of meta-document is that you could
start
> > attaching the test results to the top too, and then it goes on to
double
> up
> > as the output of the checker as a whole, complete with audit trail.
> >
> > There are probably other ways this could be done. SAX, regex, or
simply
> > parameters to a smaller set of test classes, for example. Or all of
the
> > above...
> >
> > But it's the approach itself I'm most interested in socialising at
the
> > moment. Any thoughts?
> >
> > Regards,
> > James
> >
> >
> > James Pearce
> > CTO
> > http://mtld.mobi
> >
> >
> >
> > -----Original Message-----
> > From: public-mobileok-checker-request@w3.org
> > [mailto:public-mobileok-checker-request@w3.org] On Behalf
> > Of Sean Owen
> > Sent: 05 February 2007 14:31
> > To: public-mobileok-checker@w3.org
> > Subject: Re: Some draft code for mobileOK Basic Tests RI
> >
> > That attachment was too big -- here is just the source code which is
> much
> > smaller. To actually compile it, if you want to do that, you need
> Jakarta
> > Commons HttpClient, Codec and Logging classes.
> >
> > Sean
> >
> > On 2/5/07, Sean Owen <srowen@google.com> wrote:
> > > (Does this list accept attachments -- and/or is it generally a bad
> > > idea to send big attachments to these lists?)
> > >
> > > Here's a sketch of the mobileOK Basic Test framework plus a
partial
> > > implementation of the caching test. It's enough to demonstrate
what
> > > I'm thinking and start an API discussion. Before I do anything
more or
> > > deal with dev.w3.org we should finalize the structure of the API.
> > >
> > > Sean
> > >
> > >
> >

Received on Tuesday, 6 February 2007 12:16:59 UTC