Re: types of conformance

Happy New Year, All!

I wrote:
>>In my experience, the first concern is to define "conformance of a document" 
>>...
>>Then we can talk about conformance of a tool that produces one of those. 
>>...
>>And then, maybe, we can talk about conformance of a tool that consumes one of
>>those RIF resources.  The problem with this "class of conformance" is that we
>>can't really specify what such a tool will do -- we can only specify that it 
>>must interpret the RIF resource as specified in this Recommendation, and that
>>whatever actions it takes that may be driven by that resource must be 
>>consistent with the interpretation provided in this Recommendation.  Here 
>>conformance to a dialect means that the tool is not required to accept RIF 
>>resources that do not conform to one of the specified dialect(s)/subset(s) to
>>which it claims conformance.

Sandro wrote:
> I think the situation you're describing is this: every system which
> consumes RIF handles some features, determined by the implementers for
> each release.  The release notes for each version would describe, in
> detailed technical prose (re-written by the marketing department) all
> the features they implement to some degree in that version, and maybe
> what the restrictions are on the implementations.  And books come out
> with big compatibility tables saying feature X is implemented well in
> products A, B, and C, poorly in D and E, and not at all in F, G, and H.
> (The others products on the market are not listed, and the book grows
> frustratingly out of date...)
> 
> This is what happens whenever a spec is too hard to implement, or
> doesn't define a bar for conformance.

This is not at all what I meant.  My expectation was that we would define 
specific sets of features as well-defined "sublanguages" of RIF that have 
known well-defined semantics.  E.g., a "core RIF", and "core + features X and 
Y" and "core + feature Z"  and "everything" = core + W, X, Y, and Z.  A tool 
can conform to one or more of the standard sublanguages, but not to an 
arbitrary collection of RIF features.

This is related to what Christian said:
> We also discussed avoiding trviial compliance: is requiring that all
> compliant implementations list the feature they "opted out" enough for
> that purpose, or do we require that the specification of a dialect lists
> a limited number of optional features?
> Wouldn't that bring us back to the discussion of what is the bottom line? 

The "bottom line" is what I think of as the "core" sublanguage -- minimum 
compliance.  If we have just the "core" and 12 independently optional 
features, we will have in fact defined 2^12 distinct sublanguages, and in all 
likelihood they will have no consistently definable semantics.  So I reject 
this approach.  If we have just 4 independently optional features, we will 
have 16 distinct sublanguages, and even that is probably too many.  I would 
prefer that we create a kind of partial ordering among the sublanguages, in 
which any ordered sequence has a consistent semantics, while incomparable 
branches needn't.

Presumably the RIFRAF research on target rules languages (as implemented) and 
Benjamin Grosof's master semantic model (SCLP) gives us a leg up on what 
reasonable choices for useful sublanguages would be.  It wouldn't bother me a 
lot if some of our choices would be near to what some tools can currently do, 
but beyond their current capabilities (for want of having a well-defined 
competency, as distinct from whatever feature was hacked into them last). 
This was, after all, the situation with DL reasoners when OWL was adopted.

The greater problem is in trying to define the behaviours of a conforming 
tool.  We think of a conforming tool as a reasoning engine that implements a 
useful algorithm that covers and is consistent with the standard semantics for 
a specified sublanguage.  But what of a tool that simply reads a conforming 
document and outputs the ruleset in the Jena form?  Similarly, what of a tool 
that has some user-friendly browser-based interface that deals with ontology 
matching or XML message equivalence and spits out transformation rules in RIF 
format?  These things aren't reasoning engines, but surely they are in some 
sense conforming.  That is, the ability to perform reasoning from a RIF 
ruleset is a "feature", and one I might add that we may find it difficult to 
define.

Sandro said:
> Vendors specify the mapping; we specify the features they have to map to
> -- or tell the user if they cannot -- right?

Yes.  But the features we define should come in well-defined packages 
(sublanguages) with consistent semantics.  And a tool has to conform to one of 
the sublanguages; RIF is not a Chinese menu.

-Ed

-- 
Edward J. Barkmeyer                        Email: edbark@nist.gov
National Institute of Standards & Technology
Manufacturing Systems Integration Division
100 Bureau Drive, Stop 8263                Tel: +1 301-975-3528
Gaithersburg, MD 20899-8263                FAX: +1 301-975-4694

"The opinions expressed above do not reflect consensus of NIST,
  and have not been reviewed by any Government authority."

Received on Wednesday, 3 January 2007 18:24:00 UTC