W3C home > Mailing lists > Public > public-mwts@w3.org > September 2009

Re: [widget-digsig] Test assertions

From: Marcos Caceres <marcosc@opera.com>
Date: Tue, 29 Sep 2009 12:40:52 +0200
Message-ID: <b21a10670909290340t4e105842laa21016f7ea17e7@mail.gmail.com>
To: Dominique Hazael-Massieux <dom@w3.org>
Cc: public-webapps@w3.org, public-mwts@w3.org
On Tue, Sep 29, 2009 at 9:51 AM, Dominique Hazael-Massieux <dom@w3.org> wrote:
> Hi Marcos,
> As Kai alluded to in his report [1], we had a chance to look at Widgets
> Digital Signature last week to see what would be required to create test
> cases for that specification.
> As part of that exploratory work, we started two documents similar to
> the ones that were developed for P&C:
>  * a test suite edition of the spec, at:
> http://dev.w3.org/2006/waf/widgets-digsig/Overview_TSE.html


> It marks up 17 test assertions for user agents
>  * a test plan document where these test assertions appear,
> automatically extracted:
> http://dev.w3.org/2006/waf/widgets-digsig/tests/


> We discussed (but haven't documented yet) that the test cases for DigSig
> would be of two main types:
>  * the ones testing the proper parsing of the signatures files, similar
> in the work done for config.xml in P&C
>  * the ones that focus on the actual hash/signature validation
> algorithms
> Kai took an action item [3] to start working on tests cases; that said,
> as I was the one working on marking up test assertions in the
> non-official test-suite-edition of DigSig, I noticed that DigSig seems
> much less testing-ready than P&C is (thanks to the huge efforts you've
> put in the TSE for that spec).

Yes, we've learned some hard lessons from P&C. It was a HUGE mistake
to go to CR without a test suite. It won't happen again for any future
Widget spec (well, not without a formal objection from me).

> For instance, DigSig considers signature files as class of products,
> where as these aspects would be better considered under either the
> generic user agent or the conformance checker angle; as a result, many
> of the MUST in the specs can't easily be linked to a test case in the
> current state of the spec - I only marked up the 17 ones that were
> fairly clearly testable.

Right. We need to convert those to statements of fact.

> Are you considering putting the same kind of work in DigSig as you did
> in P&C to ease the testing phase?

Yes, I assumed I would have to. I want to finish P&C first, however.
It's super close to being done.

> Could you look into the existing 17
> assertions as a starting point to see if they reflect realistically the
> expected behavior of a user agent?

Yes. I will do that. But I need a few days.

> Should you start working on a TSE for digSig, it would be great if you
> could keep the same test assertions ids I've started to use (although
> given their small number at this time, it wouldn't be a big deal if you
> choose not to); note that I opted to use two-letters longs ids (e.g.
> ta-aa, ta-ab), rather than the 8-random-letters-long ones you picked for
> P&C that made up for interesting discussions last week :) [2]

Again, more lessons learned - won't happen again. I will restrict to 2
characters, that's still a massive address space. Apologies again, mia

Marcos Caceres
Received on Tuesday, 29 September 2009 10:41:56 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 16:34:13 UTC