Re: keeping "how to run GRDDL test cases" somewhat separate

On Thu, 8 Mar 2007, Dan Connolly wrote:
> In recent minutes, I see...
>
> "*ACTION:* Chime adding sentence to Test Case Doc specifying that local 
> security policy must be set to none before running tests"
>
> I'm interested to see how that turns out; it's fine to note, in the section 
> on notes
> to implementors who might want to run the tests, that security policy
> interacts with the ability to compute various GRDDL results. In particular,
> I'd like us to work out the details of reporting, in EARL, "I didn't find
> the GRDDL result in this test due to policy/configuration".

What would be the trigger for this? Is the GRDDL-aware agent which the 
test harness drives expected to report it's policy decisions?  That is the 
only way I can imagine this would trickle down to the EARL report 
generated from a test run.

> So far, all the stuff about running the tests is incidental and informal.
> I don't want to add any RFC2119 MUST style stuff about how to test a
> GRDDL-aware agent.

You can't test functional requirements authoritatively without 'some' 
control over your environment.  This is especially the case with GRDDL as 
we have *many* factors which introduce ambiguity: faithful-infoset 
considerations, security policies, user interaction, and 
client 'capabilities'.  Some of which have direct consequence on our 
GRDDL-aware agent compliance label.

Without guidance on how to minimize the ambiguity you drastically reduce
the usefulness of having a test harness for GRDDL in the first place.

> In particular, I don't think we should advocate
> making it *possible* to set the security policy of a GRDDL-aware
> agent to "none".

I don't see this as 'advocating' the use of toggling security policies, 
but as guidance for implementors who sincerely wish to test 
compliance WRT to the 'GRDDL-aware agent' label.

I consider our mention of security policies and their effect on what GRDDL results are computed as an 
'untested hook' until we have a testable mechanism to demonstrate 
expected behavior irrespective of security policies.

> Maybe having the "how to run tests" stuff in the same document
> is too confusing.

I don't think so.  I think a paragraph will do and is best located in the 
test document. Where else would you put information on how to run tests 
other than in the document describing the tests themselves?

Chimezie Ogbuji
Lead Systems Analyst
Thoracic and Cardiovascular Surgery
Cleveland Clinic Foundation
9500 Euclid Avenue/ W26
Cleveland, Ohio 44195
Office: (216)444-8593
ogbujic@ccf.org

Received on Tuesday, 13 March 2007 20:29:10 UTC