Re: Testable assertion tagging for W3C specifications

On Wed, 29 May 2002, David Marston/Cambridge/IBM wrote:

> I want it to address them in a way that recognizes that they *are*
> productions, assertions, or whatever.

Why? What is the rationale for recognizing one "whatever" from another
"whatever" and, hence, limiting the number of "whatever"s and the
relationships among them? Can we abstract the model enough so that we
do not depend on today's "whatever" definition? I think we can.

> Based on your verbiage above, I gather that you are referring to a
> text range, delimited at both start and end.

Yes, a single "address" could be a text range or an XML fragment.

> Now, let's look at typical normative source under the current
> practice. Quoting section 7.7 of XSLT:
> 
> <li>[...]</li>
> 
> You have to go outside the enclosing <ul> and read prose to
> realize that all the above is only applicable *if* there is no
> "value" attribute set, so no sentence in the above can be quoted
> standalone.

Sure. A single test case (or whoever is quoting the specs), can use a
set or sequence of addresses to highlight/extract relevant pieces. I
think I mentioned that possibility on the list. Whether a set or a
sequence of addresses is itself an address is an open question (and
not a terribly difficult/important one, IMO).

> But you can't even quote sentences without resorting to
> substring() and relying on non-varying character counts. What you
> can isolate is a whole text node or a <code> element, so you can
> get ungainly sequences with dangling sentence fragments. For
> example, to specify the interaction of level="any" and specified
> from, without just quoting the entire <p>, you get: " axes). If
> the <code>from</code> attribute is specified, then only nodes
> after the first node before the current node that match the
> <code>from</code> pattern are considered." The above is 5
> sequential descendants of the <p>: text()[5] through text()[7].

A good addressing scheme should allow for quoting of arbitrary
sequential text. Thus, it would not leave " axes)." string in the
above example. It would address exactly what you want.

> As I mentioned earlier in this thread, this is about more than
> just a one-way link. We should try to identify the passage as
> something higher-level than just text()[5] through text()[7]. We
> should be able to say that we are pointing at the sentence that
> describes the interaction of two attributes.

A good design would separate the two aspects: extraction and
rendering/presenting/processing in a meaningful way. For example, when
a user clicks on a test case description, the description may include
some readable narrative _and_ precise quote(s) from the Recommendation
that, together, give user a good idea on what is being tested (among
other details).

In other words, "WHAT we are really pointing at" should be outside of
the pointing (addressing) scheme both because we cannot predict all
things that people will point at AND because there is no practical
need in making such predictions. The smarts should be in the test
suite or elsewhere; the addressing scheme should be simple.

You are essentially arguing that we need not simply pointers/addresses
but pointers with types. Introducing "pointer types" is only useful if
you start manipulating with pointers of different types (i.e., doing
address arithmetic for pieces of a Recommendation) -- hardly on the
horizon for most test suites!

> I mentioned before about discerning that two tests address the
> same interaction of attributes. If that is too adventurous for
> now, would you at least agree that we ought to be able to take a
> collection of tests, follow their citation links, and see how much
> of the verbiage is covered by the tests? (Picture a Recommendation
> with all "covered" sentences having a light green background. You
> could eye-scan the Rec, applying human intelligence, to see which
> of the uncovered portions look like testable sentences.)

Yes, absolutely! Moreover, my team is working on that kind of rendering
of RFC 2616 for our test suite. As far as I can see, the proposed
simple addressing approach is sufficient to build a Coverage Map given
a set of addresses derived from a set of test cases.

> > My understanding is that no tag taxonomy is needed to address pieces 
> > of text or XML. We are not talking about the process of automatically 
> > locating assertions in a random Recommendation, are we? While fun, 
> > that task seems to have little utility.
> 
> This is the pivot of our disagreement. To me, "addressing pieces of
> text" is the activity that has little utility. 

I have formulated my requirements, which were derived from my test
suite needs and my understanding of other's needs. I think that if I
can address any arbitrary piece of a Rec, I can do everything I need.
I can specify more detailed requirements (e.g., persistency of
addresses), but I do not want to waste your time until we agree on the
principle.

What are your requirements? If we start from requirements, we can come
to an agreement much sooner. So far, a simple "piece extraction"
scheme seems to cover all your explicit needs.

> We *may* want to locate assertions in any Rec, but (as you say) we
> are more likely to want to locate assertions in designated Recs.
> It's more than just test suites in the overall plan, though. We
> already have some Recs citing other Recs down to the level of
> individual productions, which is beneficial. Recs should be
> citable at the granularity of any sentence, equation, or similar
> construct that normatively expresses a complete thought.

Exactly. I think that the proposed approach can be used to implement
all of the above. If you can address any arbitrary piece of a
Recommendation, then you should be able to address any complete
thought, provided the Recommendation has complete thoughts.

On the other hand, once you start introducing artificial barriers
such as hard-coded tags/types, you will run into problems of citing
thoughts that cross those artificial barriers (e.g., a paragraph can
contain many interleaved thoughts in an unstructured format)!

Alex.

Received on Wednesday, 29 May 2002 18:10:01 UTC