Re: Formalizing testSubject

[...]
> The actual requirement for evaluation transactions is to
> preserve enough information about what was evaluated
> so that another party can reproduce your results.

Agreed, and well-put.

> The QA group should be consulted on this point.

Noted.

> For a web page one wants to retain:
> all headers in the HTTP transmission by which received.

Coincedentally, Jonathan Borden just published a modelling of HTTP
transactions in RDF.

   http://www.openhealth.org/xmtp/HTTP
   Modelling HTTP in RDF, March 1, 2002

Although he uses the RFC's IANA URI as a namespace which I consider to
be in error, it's a vocabularly that may be worth using in EARL.

I love (isl: hrífast) the idea of having a transaction metadata
property cascade for query engines, but doing "else if" in declarative
queries is not all that easy, in my experience. I also don't think
it's a true one dimensional cascade: you may want to query according
to several of the properties, for example by location, the HTTP last
modified date, *and* the actual local timestamp.

> [...] it will either be an entire verbatim copy of the resource
> representation as received in the HTTP transmission or some
> hash or checksum derived from it

Aside: or some checksum derived from a part of the representation, as
per Nick's experiments.

> Not a precise enough identification to isolate situations where
> different evaluators reach different conclusions because they
> obtained different [concrete forms] of a resource under the
> same URI and timestamp conditions.

Fair enough... but on the same page, we can't be too specific as to
the type of information that is attached for disambiguation; e.g. we
can't force people to use hashes.

> The bane of test programs is the "re-test OK" case, [...]

Which may well have been caused by a rigidity in the test case
frameworks - not allowing for extra information because it would bork
the mechanism, and cost too much money. The fact that we're using a
decentralized/Webized format doesn't solve that problem - even though
creating a new property is just a matter of minting a new URI for it -
because deploying the changes still takes effort. But the
decentralization might help someone, somewhere.

> Any and all details of the actual data or transformation
> as it existed in the concrete test transaction are of interest
> for diagnostic purposes.

Good point.

> If the supplier of the web page or tool has computed
> a Content-MD5: value [...]

SHA is genrally considered safer for huge databases, and since EARL
promotes test cases being stored on a global scale, perhaps SHA-1 is
not secure enough (although there have been no noted collisions to
date).

> [...] the testing activity should compute a hash of their
> own chosing that will be highly resistant to false compares
> between different evaluators of the same test subject.

As a UUID, you mean?

> [...] this projection or view transform should be handled as
> information about the test applied and not as identification
> of the subject to which the evaluation was applied.

Ooh, stunningly good point, but what does this mean in terms of the
EARL model/vocabulary? I wanted to create classes that let the
assertor [1] of the evaluation tell people how they interpreted the
test subject with regards to their particular test. In other words, I
think that the EARL test subject sub classes are too objective to
merit your "police car" example as being a valid analogy.

[1] Archaic? But Kant, R.S.Candlish, and Prior seem to have used it
according to a cursory Google search. "er" seems counter intuitive to
me, and not as phenomic. And then there's Aaron's wonderful
intentional misquoting of Thomas Macaulay: "<AaronSw> The assertors of
liberty said [assertor was] not a word." OTOH, we could have used
"Rsigosrig": it wouldn't have mattered.

P.S. At the moment, there are 987 results for "assertor" on Google,
and 7 of those on the front page are EARL-related :-)

--
Kindest Regards,
Sean B. Palmer
@prefix : <http://purl.org/net/swn#> .
:Sean :homepage <http://purl.org/net/sbp/> .

Received on Sunday, 3 March 2002 21:57:51 UTC