Re: Testable assertion tagging for W3C specifications

On Thu, 30 May 2002, Dimitris Dimitriadis wrote:

> > In other words, (and that's the gist of this e-mail): perfect spec
> > markup surely solves the addressing problem, but we must not assume
> > that all real markup is or will be good. A good addressing scheme
> > should work well with perfect and real specs.
> >
> [dd] Well, insofar as we can have say on the way the markup you refer to 
> above is written and used, I think we can certainly assume that we will 
> solve the adressing problem, as we will have though of it when designing 
> the Spec authoring schema. If so, the adressing scheme would work with 
> all specs written from one point in time and onward.

I am afraid this logic is false. Markup, even a perfect one, will not
mark every single character. On the other hand, we cannot and should
not limit citation abilities of test suites and such. Thus, markup
alone cannot solve citation/addressing problem because I may want to
cite something that was not explicitly marked in the spec.

> > Sure. I am not arguing against better markup. I am arguing against a
> > close relationship between "better markup" proposed by QAWG and
> > "better addressing" proposed by QAWG. In other words, better
> > addressing should not be done through better markup (because strong
> > markup dependency limits addressing abilities no matter how good that
> > markup is).
> >
> [dd] Could you elaborate on this? Why would structured markup be 
> counterproductive?

Structured markup is not counterproductive, but relying on structured
markup alone when designing an addressing scheme limits your options.

For example, let's say all test assertions are isolated, marked, and
identified by a unique number. Great. Now a test suite can address any
assertion with ease. However, if the addressing scheme relies 100% on
those unique IDs, that addressing scheme does not let me cite a
non-normative piece of text OR cite a part of the assertion! I may,
for example, believe that what the authors considered to be an atomic
assertion is, in fact, two assertions, and I want to provide the
authors with one test case for each [sub]assertion to illustrate/prove
my point...

> [dd] I think "extra burden" arguments could be dismissed or argued 
> against in any case, given that the "extra burden implies better 
> quality/addressability/what have you" has been clearly demonstrated. I 
> think that's where we should focus our attention.

I do not think that it is possible to demonstrate that an extra burden
of a yet-to-be-invented perfect markup implies [better] anything. That
is why I said that the arguments can be dismissed [only] if the extra
burden is 100% optional...

> > I agree, as long as QAWG can keep "QA Tool Collection" 100% optional.
> 
> [dd] The QA WG was formed to aid Working Groups in producing high 
> quality and testable specifications. On the question of whether this aid 
> will result in optional or mandatory tools, it is at much a question of 
> design as of "marketing". If teh QA WG manages, with help from 
> interested parties, to come up with a set of tools that are generally 
> accepted, I don't think there's an issue. And, finally, making people 
> accept a particular set of tools can be as much an issue of education as 
> of anything else.

True. Just keep in mind that having anything mandatory increases (or
should increase) your burden of prove that whatever you mandate is
always better than the alternatives and, hence, mandating is better
than leaving it optional.

> > My current intention is to have an addressing scheme where each
> > address has two components: context and citation. The context
> > component will locate the area of the spec that contains the citation.
> > There MUST be a single match for all context searchers for the address
> > to be valid. The citation component is the actual text (or markup)
> > being cited (within given context). The open question is how to
> > represent context and citation to allow for mathes to be precise but
> > flexible enough.
> >
> [dd] Wouldn't a precise pointer provide a (local) context, or wouldn't 
> that be enough for our needs?

Well, it depends on what your pointer is. If your pointer is simply a
piece of cut-and-pasted text, then it may not be precise enough to
determine context. If your pointer is a byte offset, then yes, it
determines context precisely but has some bad properties like
dependence on unrelated document changes or CRLF interpretations.
These are not the only two pointer options, of course.

At this time, I do not know what the best pointer architecture is.
Hopefully, the XPath/Pointer/Query gurus on this list can suggest good
candidates that meet basic citation requirements I have outlined
before.
 
> > I suggest that the person citing is given both a good citation tool
> > and specs with good tags. Since we cannot control the spec format or
> > author's intent, and since there are old specs, my approach seems to
> > be much more practical than relying on authors to produce markup
> > perfect for citing. Not to mention cases where you want to cite
> > something that the author did not intended to be cited (by author's
> > mistake or design, does not matter).
> >
> [dd] I think this is one of the most important issues raised here. Let 
> me spell is out (and Alex, correct me if I'm wrong):
> 
> 1. We cannot control the spec format
> 2. There are old specs

3. We cannot control the authors use of the spec format
4. We should not control all possible [future] citations of the spec
 
> The issue of adding burden is perfectly valid as fas as old specs
> are concerned. What I whink, and hope that most people agree with,
> is that we should not look at new specs and old specs and think
> they are that similar. Given time, we'll come up with a set of
> tools that indees give control over the spec format. This in turn
> renders the second point above uninteresting for those newer spec.

Let's forget about old specs for now. You argue that QAWG can design a
perfect format for the new specs. Let's assume it has happened. Still,
your addressing scheme should not rely on that markup format alone
because of the requirement to cite unmarked pieces of the spec. See
above for an example.

> > Again, good markup helps, but
> > we should not assume that all markup is good!
> >
> [dd] You're absolutely right, and that's why we wan to ensure that it is 
> in the future.

Good markup helps, but is not a complete solution. You still want an
addressing scheme that can address unmarked text or unmarked XML
fragments.

While they live in the same large problem domain, the requirements and
development of good markup formats are separate from the requirements
and development of good citation schemes.

Alex.

Received on Thursday, 30 May 2002 18:21:25 UTC