Re: [widgets] Test Suite for Packaging and Configuration

On Fri, Aug 7, 2009 at 8:47 PM, Scott
Wilson<scott.bradley.wilson@gmail.com> wrote:
> Hmm, well as you really can only test observable behaviour - and the
> behaviour of a widget is really what the A&E spec is concerned with... I can
> see the problem we have here.

Yes. However, making config doc values available to the API is not the
only means to do this testing. Firstly, as "implementers" we have
complete control over the implementation, so we can check the internal
state at any point of execution (and build tools to facilitate such
testing). The A&E is really meant for authors, not for testers. It
just happens that some of the values in the config document end up
being available via the A&E spec. Of course, when we create the test
suite for the A&E, then we must check that the values derived in from
the P&C correctly end up bound to the A&E's widget object.

Secondly, although some elements are not exposed as part of A&E (e.g.,
icon and license), there are still conformance requirements as to how
the user agent is supposed to expose the content of those elements to
the end-user. For example, for the license element, the spec says:

"When the href  attribute [of the license element] is used, the user
agent should provide users the ability to view or otherwise link to
the referenced license."

Although not mandatory behavior, nor exposed via A&E, it is still
manually testable ("test: does the widget provide some means to show
the license?"). Having said that, we could have made this more clear
in the spec by saying: "Upon request by an end-user, a user agent
should expose a license in an accessible manner (e.g., visually or
aurally)."

> If we completed the "widget catalogue" Atom feed profile we mooted a while
> back that would also give us another behaviour to test with (as in, you can
> ask a UA "what widgets do you have?" and check the catalogue metadata fits
> with the test cases).

Right. That would have been good. I imagine as widget galleries begin
to appear, we will quickly find out if we have packed enough metadata
into P&C. We did a lot of homework into widget metadata (i.e., the
landscape doc and my PhD), so I'm fairly confident we covered most use
cases. But the proof of the pudding is in the eating, as they say:)

> It could be a good sanity check - if an element in P&C has no effect on any
> behaviour that we can observe, what is it there for? Should it be exposed in
> A&E? Or in a widget catalogue feed?

Agree. The expected usage is given by both the definition of each
element and the requirement from which the element was derived. It
might be worth doing an editorial tightening up of definitions where
things are not clear. I'm always open to suggestions.

Kind regards,
Marcos

Received on Monday, 10 August 2009 09:02:00 UTC