Re: Formal Objection: publication of XForms 1.1 as LCWD

Hi Jan,

> Moreover, we are in fact actively maintaining the new test suite, and
> welcome any further input that would help us improve it.

I do have one suggestion, which is to try to write tests in such a way
that they can be automated. (I haven't copied this to the editor list,
since it's obviously not an LC comment, just a 'nice to have'.)

For example, one of the tests in the 1.0 test suite aims to ensure
that there are never two instances of the same submission in progress
at any one time. The test uses XForms messages, which require a user
to interact with the form, but if we used some instance data and a few
judicious xf:setvalues in the event handlers for submission, we should
be able to create a form that has a simple output--maybe saying
nothing more than 'success' or 'failure'--indicating whether two
instances were in progress or not.

To illustrate the idea, here is a test for the new XForms 1.1 @resource feature:

  <http://svn.x-port.net/svn/public/samples/specifications/xforms/1.1/submission-resource.html>

It uses submissions to Flickr to obtain lists of photos using various
tags. (We don't do anything with the images referred to, but it's a
convenient source of predictable XML.) Each test has an associated
piece of instance data that is used to keep track of the progress of
the test, so as the submission progresses a string is built up.

For example, when xforms-submit is received the result string has
'submit' concatenated to it. Then when xforms-submit-done is received,
the string has '|done' added, and the data is then checked to see if
Flickr returned a success code--if so '|dataok' is added to the
string. (xforms-submit-error would obviously add '|error' to the
string.)

You'd expect the resulting string to be 'submit|done|dataok', but you
could get 'submit|error' if there was a problem with the request, and
perhaps even things like 'submit|dataok' (if the xforms-submit-done
event didn't fire correctly) or even '|donesubmit' (if events happen
in the wrong order). Whatever you get, if it isn't
'submit|done|dataok', then the test has failed.

The good thing about this is that we don't need to have a text
description of what a user is supposed to see, and we don't have
messages that need to be acknowledged. In fact, you'll see in the
example I have linked to, that we initiate the test on xforms-ready,
so the tester need do nothing other than load the form and see if the
expected string appears (having been placed in an xf:output). And by
having the string displayed, rather than messages, we can easily test
with a tool like Selenium, or one of the many other unit-testing
systems out there.

I think it would be great to produce a test-suite that people can use
to test their implementations on a regular basis. I remember running
the tests when we were getting XForms 1.0 ready for last call, and it
was painful! There is so much manual intervention that it's not
something you'd run very often. But if we can automate this, it's the
kind of thing that both implementers and end-users could use on a
regular basis.

Any thoughts?

All the best,

Mark

-- 
  Mark Birbeck, formsPlayer

  mark.birbeck@x-port.net | +44 (0) 20 7689 9232
  http://www.formsPlayer.com | http://internet-apps.blogspot.com

  standards. innovation.

Received on Friday, 27 April 2007 10:12:26 UTC