Re: story telling and test cases (offlist)

On Tue, 2007-03-27 at 23:34 +1000, Lachlan Hunt wrote:
> Dan Connolly wrote:
> > I think I chose poorly when I picked the subject
> > "brainstorming: test cases, issues, goals, etc."
> > 
> > What I really want is test cases; i.e. concrete documents
> > sent as attachments.
> 
> Test cases for what?

We have enough mail in public-html without the chair repeating himself,
so I changed the cc from public-html to www-archive.
  http://lists.w3.org/Archives/Public/www-archive/

The answer I gave is...

http://lists.w3.org/Archives/Public/public-html/2007JanMar/0045.html
[[
Or maybe it's a test case that characterizes an issue, like "the HTML 4
spec
says this document is no good, but clearly it is, according to current 
practice."

Or perhaps it characterizes an interoperability issue, a la "browser X works
as I'd expect with the attached document, but browser Y doesn't."

Perhaps you have a goal for this WG; I'd like to think you can make up
a document that is relevant to that goal; e.g. if you want some feature
that's not in HTML 4, attach a document that uses the new feature,
and motivate it in the body of the document. Maybe the motivation
will naturally come in the form of a story or use case.

Likewise requirements... if you have a requirement, tell us about it
and attach a document that does or doesn't meet the requirement
(or shows that some software does or doesn't meet the requirement).
]]


>   Test cases don't really help when there's no 
> structure.

I think there's a reasonable chance that structure will emerge.
That's pretty common group behavior.

>   Sure, we could all submit random test cases testing random 
> features, but that doesn't really help anyone.

It helps me. I like to study concrete examples.

Somebody finally sent a concrete test case for an XML Input
Control.
 http://lists.w3.org/Archives/Public/public-html/2007JanMar/0585.html

We can now all objectively measure how much
software supports it. (Zero). Then we can talk about the
cost of getting it ubiquitously deployed, which brings
us back to story telling (use cases, problem statements, etc.)
If the guy is willing to step back from his solution
looking for a problem, we can discuss existing mechanisms
that might meet the need; again, if we attach (or point
to*) concrete examples, he can objectively measure whether
they meet his requirements.



> Good test cases should test a specific feature and have clear pass/fail 
> conditions.  For mroe information about writing test cases, see the CSS 
> 2.1 test case authoring guidlines [1].  There's some good information in 
> there, just ignore the CSS specific stuff.
> 
> The example "test cases" you mentioned can't really be considered test 
> cases.  They're just demonstrations of features because neither 
> expressed any pass/fail conditions.

Yes, calling them "test cases" is generous. I get tired of
writing "test case sketch" or "test case input" though.

I agree that there's a lot of work to turn these examples into test
cases. I am hoping someone will step forward to do that work.

> So the question is, precisely what do you want test cases for?

To make people think before posting, mostly. Asking for use
cases works similarly. Hence the subject "story telling and test cases".

>   What's 
> your ultimate goal of having them?  What will make them useful for you 
> specifically in this case?
> 
> [1] http://www.w3.org/Style/CSS/Test/guidelines.html#key-aspects
> 
-- 
Dan Connolly, W3C http://www.w3.org/People/Connolly/
D3C2 887B 0F92 6005 C541  0875 0F91 96DE 6E52 C29E

Received on Tuesday, 27 March 2007 14:02:24 UTC