Re: Tools / process for building the MathML test suite?

Robert,

Thank you for the explanation. I have been through a
similar experience with my document production (perl)
scripts: I don't consider them very portable or exportable,
even though I would like to share the ideas with other
editors. And I don't have time to turn them into a general-purpose
production system (even one of small scale). 

I would be interested in seeing what you've done, as I like
the results. I won't expect any support of the scripts.
If we require images at all, it will be as part of content,
not pictures of what the results should look like (as we 
have functional requirements unrelated to specific rendering
or user interface look and feel).

I am also looking into how other W3C Groups with test suites
(CSS, DOM, SVG) manage theirs. 

Thanks for responding so quickly,

 - Ian

Robert Miner wrote:
> 
> Hi Ian,
> 
> > I am wondering what process the MathML WG adopted
> > to create the test suite. For instance, was there a
> > particular process followed to submit/approve tests?
> > What tools were used to create the test files or
> > to generate the test suite?
> 
> Neil Soiffer at Wolfram Research and I mostly put together the MathML
> test suite, with a few other people contributing a test here and
> there. Unfortunately, the tools we used are not portable.
> 
> I wrote an ad hoc CGI-based system for submitting tests, either one at
> a time, or zipped together in a batch.  We defined a little XML
> vocabulary for a test -- author, description, spec section, rendering,
> MathML code, etc.  Then I wrote some scripts for adding, deleting and
> modifying tests.  I hooked up DSI's WebEQ MathML software on the back
> end to generate images of renderings if they weren't submitted as part
> of the test.  Finally, there were some scripts for regenerating the
> table of contents, for zipping the whole suite up, etc.
> 
> It was all custom Perl code, very much tied to our specific test
> suite structure, and I could only swing it because I hosted it on one
> of our company servers where I had complete control over permissions,
> etc.  There was also plenty of hand work with Perl, find, grep, etc
> that I just coordinated with modifications to the scripts.
> 
> It worked okay, though it suffered from growing pains, since I was
> building the management system at the same time as the test suite.
> 
> Unfortunately, the system is pretty much useless now.  Once the suite
> was "done" we moved it to the W3C servers, where there was essentially
> no hope of getting enough access to continue to maintain the scripts
> -- for example WebEQ needs access to an X server for graphics calls,
> and there were a bunch of other insecure aspects to it.  I think it
> would really have to be done by one of the web admin guys there.
> 
> Further, once the suite was at W3C, we started having to make changes
> by hand, to handle bugs and errata, and it just got out of control to
> the point where it would take some significant effort on my part to
> reimport into the development environment.
> 
> However, Neil and I are tasked with doing a major round of
> additions/corrections/reformatting soon, and so I have been wringing
> my hands about how to best go about it.  I really just don't have time
> to do the programming for the forseeable future, but I would gladly
> zip up the stuff I have and send them off to anyone who is interested
> in taking over and putting in the time to make it a really robust,
> general purpose system.  If you pulled out the WebEQ part, and just
> stuck with requiring any images to be submitted along with the
> associated test, that would clear out one of the big obstacles.


-- 
Ian Jacobs (ij@w3.org)   http://www.w3.org/People/Jacobs
Tel:                     +1 718 260-9447

Received on Thursday, 20 September 2001 18:38:20 UTC