RE: Questions and proposals related to strawman schema from F2F

1. Severity (new)
At the F2F we had consensus that a severity property, similar to
confidence and validity, would be a good thing.  I propose we add
something to the strawman.  Refer to Ian's proposal:
http://lists.w3.org/Archives/Public/w3c-wai-er-ig/2002Jun/0044.html

Rob: I am not sure how important this is, unless it supports custom
values for severity if it does not it is not a good thing. I can imagine
the discussions on what is Partial! Too ambiguous. No it would be
nothing to code something to read total results of criteria for a pass
and have items auto calculate % of Tests that passed or Failed but. The
Percentage itself could be meaningless.



2. repairInfo, expectedResult, suspectAgainst (from 1.0 test) These seem
to be evaluation-and-repair-tool-specific and don't seem necessary to
combine or compare results of tests.  No changes to the strawman
proposed.

Rob: But looking forward this is good stuff, but we can always extend
EARL on our own.

3.  operator, operatorInstructions, purpose (from 1.0 test) These
describe properties of the testCase.  Since we do not want EARL to be a
pseudo-test description language, it makes sense to get rid of these.
One question: what if the operator is different from the assertor?  It
seems the two could share the assertion. No changes to the strawman
proposed.

Rob: Why not just use Asserter and if the Asserter is the Tool, and you
want to save the operator of the tool that is simple. I Agree to Remove.

4.  testMode (from 1.0 test)
testMode had 3 instances: automatic, heuristic, and manual.  One of the
original features of the language was the ability to store information
about how the testSubject was evaluated and who made the assertion.
We get at this somewhat with properties of Assertor, but this could be
different information than the Assertor properties.  If not covered by
Assertor properties, I propose we add something to the strawman. 

Rob: Please note how it can be different so I can Understand more
clearly.

5. TestCriteria,  suite, level, excludes (from 1.0 test)
While these describe pieces of the testCase, at the face to face we
agreed that we need some way to group tests.  Suite, level, and excludes
were created to do this.  I propose we add something to the strawman.
Ideas?

Rob: Why not a Job Number???  



6. os, version (from 1.0 test)
Is this covered by platform.  It seems to me that these are necessary to
uniquely identify user agent test subjects.  If not covered by platform,
I propose we add something to the strawman.  

Rob: OS Is Cool, I would think we really need is the Environment for
Test Subject and For Test as Separate Items

7. snapshot  (from 1.0 test)
I think this was created to help deal with dynamic web content.  A
snapshot could capture the content at the time of an evaluation. Is this
handled by reprOf?  At this time, no changes proposed.

Rob: I would agree, The Test Subject and How obtained is easy enough to
keep, But if we start encapsulating files we will run into impressive
concerns of disk space usage, considering That many servers that serve
dynamic content do weird things with file sizes. Even if it was obtained
by say a QuickTest Pro Test you could provide linkage to the test.

8. date  (from 1.0 test)
I believe SBP created an EARL-specific date datatype to deal with
ambiguities in DC. Date is important in identifying "versions" of
dynamic web content.  The strawman does not contain any constraints
other than rdfs:range and rdfs:domain, whereas 0.95 and test1.0 had a
variety of constraints (mostly daml).  Many folks felt that constraints
was something to attempt in a future version after daml/oil/owl/other
are stable.  Is creating a datatype for dates a similar issue?  Has this
been dealt with someplace else (DC?) since earl:date was created?  I do
not have a proposal one way or the other at this time.

Rob: I Agree

Received on Wednesday, 21 August 2002 18:40:19 UTC