[widget-digsig] Test assertions

Hi Marcos,

As Kai alluded to in his report [1], we had a chance to look at Widgets
Digital Signature last week to see what would be required to create test
cases for that specification.

As part of that exploratory work, we started two documents similar to
the ones that were developed for P&C:
 * a test suite edition of the spec, at:
http://dev.w3.org/2006/waf/widgets-digsig/Overview_TSE.html
It marks up 17 test assertions for user agents
 * a test plan document where these test assertions appear,
automatically extracted:
http://dev.w3.org/2006/waf/widgets-digsig/tests/

We discussed (but haven't documented yet) that the test cases for DigSig
would be of two main types:
 * the ones testing the proper parsing of the signatures files, similar
in the work done for config.xml in P&C
 * the ones that focus on the actual hash/signature validation
algorithms

Kai took an action item [3] to start working on tests cases; that said,
as I was the one working on marking up test assertions in the
non-official test-suite-edition of DigSig, I noticed that DigSig seems
much less testing-ready than P&C is (thanks to the huge efforts you've
put in the TSE for that spec).

For instance, DigSig considers signature files as class of products,
where as these aspects would be better considered under either the
generic user agent or the conformance checker angle; as a result, many
of the MUST in the specs can't easily be linked to a test case in the
current state of the spec - I only marked up the 17 ones that were
fairly clearly testable.

Are you considering putting the same kind of work in DigSig as you did
in P&C to ease the testing phase? Could you look into the existing 17
assertions as a starting point to see if they reflect realistically the
expected behavior of a user agent?

Should you start working on a TSE for digSig, it would be great if you
could keep the same test assertions ids I've started to use (although
given their small number at this time, it wouldn't be a big deal if you
choose not to); note that I opted to use two-letters longs ids (e.g.
ta-aa, ta-ab), rather than the 8-random-letters-long ones you picked for
P&C that made up for interesting discussions last week :) [2]

Dom

1. http://lists.w3.org/Archives/Public/public-mwts/2009Sep/0009.html
2. http://twitter.com/dontcallmedom/status/4311968310
3. http://www.w3.org/2005/MWI/Tests/track/actions/82

Received on Tuesday, 29 September 2009 07:52:22 UTC