RE: Votes to advance documents to LC and CR

Hi!

In my opinion, these "typical exit criteria" only make sense for "typical"
W3C specifications: For languages such as HTML or XML it will generally be
justified to require two working proof-of-concept implementations within
the few weeks after the release as a CR. However, for OWL (1 or 2), I
consider the situation to be very different, because implementing an OWL
reasoner (DL or Full) is inherently difficult and non-obvious.

Imagine we wouldn't be in the convenient situation that there are already
several well-working reasoners for most of our profiles, with many
implementers being participants of the WG. What could we realistically
expect from the CR phase to receive as implementation experience from third
parties? Hardly anything, I guess, unless we extended the CR phase to a very
long time.

This has nothing to do with OWL /2/. If we were now in the situation to create
a first "web ontology language", only with the features of OWL 1 or maybe even
less, I believe there wouldn't be a big chance to get much relevant feedback
from the "industry", compared to the quality that one could expect for HTML
and friends, unless there are implementers around with exceptional expertise
in computational logics having almost-ready implementation waiting.

For the RDF-Based Semantics, as all of you will know, it will be particularly
problematic to "show a complete implementation" (not to mention two of them),
or to only have every feature readily implemented by some implementation. Of
course, the situation for OWL 2 Full hasn't become worse compared to OWL 1.

You will possibly imagine where I stand in this question: I am against having
any unrealistic exit criteria for OWL 2, and in particular for the RDF-Based
Semantics. Looking at my current work-in-progress test suite, I consider it
likely that a non-fake implementation can succeed on most (80%?) or even all
of the test cases in a not so far future, but hardly within the scope of the
upcoming CR phase.

(Faking is, of course, always trivially possible, by either hard
encoding all tests, or by declaring every test to be "extra-credit",
and so the exit criteria could still be met, but...).

Maybe the process document is not flexible enough for complex languages such
as OWL. Perhaps, a WG should be allowed to explicitly mark a specification
as "complex", and in such a case (if public and AC reps buy this) it will
be sufficient to "show convincing evidence (including argumentation) that
significant parts of the spec will be implementable in reasonable time and
with reasonable effort".

The bottom line is that I would prefer to have rather weaker than stronger
exit criteria of the discussed form for OWL 2.

Regards,
Michael

>-----Original Message-----
>From: public-owl-wg-request@w3.org [mailto:public-owl-wg-request@w3.org]
>On Behalf Of Sandro Hawke
>Sent: Tuesday, May 19, 2009 3:48 PM
>To: Jim Hendler
>Cc: public-owl-wg@w3.org
>Subject: Re: Votes to advance documents to LC and CR
>
>
>> we have been trying to decide how we will vote on these -- I'd like to
>> remind the WG that a move to CR includes a decision about the specific
>> exit criterion -
>...
>> p.s. for those who are looking for this in the process document, "exit
>> criteria" are known as "Criteria for entrance to Proposed
>> Recommendation"
>
>As I read the Process Document, it doesn't actually say this, but you're
>right that in practice it's something we should probably do.  The guide
>for organizing a recommendation track transition [1] says we should
>decide, among other things:
>
>    Are there any implementation requirements beyond the defaults of the
>    Process Document? For instance, is the expectation to show two
>    complete implementations (e.g., there are two software instances,
>    each of which conforms) or to show that each feature is implemented
>    twice in some piece of software?
>
>As I recall, in WebOnt (OWL 1) took the latter option, and framed it in
>terms of test cases instead of features.  That is, for each
>non-extra-credit test, there had to be at least two implementations
>reporting passing that test.
>
>I like that approach for us to use, too.  It may be that we need to
>approve some more tests, to make sure we have at least one test per
>feature.  I don't think we need to do that before entering CR (WebOnt
>was approving new test cases all through CR), but we'll need some quiet
>time at the end, for folks can try to pass the newest tests.
>
>And perhaps we should say something about profiles, too?  I wouldn't
>mind us saying that for each profile there will be two "native"
>implementations, two systems implementing that profile and taking
>advantage of it being less than DL.  (They could take advantage in
>whatever sense they like -- performance, ease of implementations, etc.)
>I don't think we need to tie that to test cases; it would just be that
>our implementation report [2] will have at least two entries for each
>native profile (DL, EL, QL, and RL).
>
>Is that what you're looking for, Jim?
>
>> and we think that may influence our decision making -
>> when will the proposed CR exit criteria be discussed/published?  We
>> might (and I stress might) be willing to abstain, as opposed to
>> objecting, to some documents depending on the specifics of these
>> criteria - if the upcoming vote is just on whether we believe LC has
>> been successfully done, that is one thing, but a formal move to CR is
>> another, and these should be discussed.
>>   -Jim Hendler
>>    AC Rep RPI
>
>Is there some reason to conclude LC without simultaneously moving to CR?
>I'm not sure what that would mean.
>
>Can you express, in a sentence or two, the core of the objection you're
>considering?  In particular, is it about the technical design of OWL 2
>-- some language feature that's hard to implement, not motivated, etc --
>or about the user base and market?  Even more in particular, what do you
>realistically think the WG could do to address your concerns?
>
>     -- Sandro
>
>
>[1]
>http://www.w3.org/2005/08/online_xslt/xslt?xmlfile=http://www.w3.org/200
>5/08/01-
>transitions.html&xslfile=http://www.w3.org/2005/08/transitions.xsl&docst
>atus=cr-tr#transreq
>[2] http://www.w3.org/2007/OWL/wiki/Implementations