Re: A simpler proposal for reporting SHACL results - ISSUE-51

Quickly getting back to the topic of whether we need severity levels, we 
have an approved requirement captured as R10.1

http://www.w3.org/TR/2015/WD-shacl-ucr-20150414/#requirements-1

/"The language should allow the creation of error responses that can 
include severity levels as desired."/

So I believe the simplification proposed by Peter takes it a bit too far.

Whether these severity levels also need to be extensible so that users 
can add their own ones is indeed questionable, and I can see arguments 
going both ways.

Minus: Complicates the spec and learning curve
Plus: Provides more flexibility
Minus: Requires looking at shapes graph, including tracking back to its 
original definition when shared online
Plus: The ordering (Info < Warning < Error) needs to be specified 
anyway, so we could just as well generalize it.

A work-around to the first "Plus" is to attach this extra information to 
the sh:Shape and sh:Constraint objects themselves. For example

ex:MyShape
     a sh:Shape ;
     sh:property ex:MyPropertyConstraint .

ex:MyPropertyConstraint
     a sh:PropertyConstraint ;
     sh:predicate ex:property ;
     sh:minCount 1 ;
     sh:severity sh:Info ;
     ex:myLocalSeverity ex:DebuggingInfo .

Each result node can have sh:sourceShape and sh:sourceConstraint, e.g.

ex:Violation_0198_2838_773
     a sh:Info ; ...
     sh:sourceConstraint ex:MyPropertyConstraint .

which makes it possible to collect additional application-specific 
information by following the links. In addition to severity, I can 
imagine some people may want to track authorship of the constraints.

With this work-around, my preference is now to leave it simple and only 
include sh:Info, sh:Warning and sh:Error, and nothing else (assuming we 
create a separate mechanism for the exceptions previously captured by 
sh:FatalError).

Holger



     ]

On 8/5/2015 10:24, Holger Knublauch wrote:
> On 8/5/2015 2:54, Peter F. Patel-Schneider wrote:
>> Here is a proposal for handling SHACL errors and validation results.
>>
>>
>> An invocation of a SHACL processor is an attempt to perform zero or more
>> validation tasks.
>>
>> Errors:
>>
>> A SHACL processor can encounter errors in trying to perform these tasks,
>> such as
>> - communications problems, e.g., the inability to access a required 
>> document
>>    or service;
>> - incorrect syntax, e.g., a syntactically malformed SHACL construct;
>> - incorrect control information, e.g., trying to start validation 
>> using a
>>    node that is not in the data; and
>
> As an aside on the last sentence, why should the system report an 
> error if a node is not in the data? I believe this is a perfectly fine 
> use case - a focus node may simply not have any triples, and this may 
> be exactly the condition that a constraints wants to check.
>
>> - resource exhaustion, e.g., exceeding allowable processing time.
>> When an error is detected the SHACL processor MUST signal the error, 
>> using
>> means different from reporting violations, and MAY terminate its work at
>> that point.  The processor MAY report on the violations that it had
>> discovered before it encountered the error.
>
> I don't think any of this contradicts the proposal by myself and 
> Dimitris. The main difference appears to be that we call Failures what 
> you call Errors. It is also perfectly fine to terminate on the first 
> Failure/Error. This would be a setting in the engine.
>
>> Violations:
>>
>> The execution of a validation task results in a number of 
>> violations.  If
>> there are no violations then the task is a success, otherwise it is a
>> failure.  Each violation is reported in a simple minimal form that is
>> suitable for futher processing.  There are are no sub-types of 
>> violations
>> such as warning violations or error violations.  There are no severity
>> levels for violations.
>>
>> The execution of zero or more validation tasks is performed by executing
>> each of the tasks in some unspecified order.
>
> The difference here appears to be that you don't want severity levels, 
> and your "Violations" are called "Validation Results" in our proposal. 
> The latter was chosen because some results may be INFO or DEBUG level, 
> making "Violation" not a good term.
>
>>
>> Discussion:
>>
>> Fine-grained control and reporting can be done by using separate 
>> invocations.
>> If some violations are errors and some are only warnings, they can be 
>> run
>> in separate invocations and processed differently by whatever calls the
>> SHACL processor.  If there are multiple severity levels for errors, then
>> each severity level can be run in a separate invocation.  If there are
>> multiple validation tasks that each need to have violations handled
>> differently, then each can be run in a separate invocation.
>
> Unless I have misunderstood you, you have contradicted yourself. 
> Further above you state you don't want severity levels, and here you 
> write about multiple levels. Note that in our proposal, each 
> constraint can have an individual severity level using sh:severity 
> (which defaults to sh:Error). This information can already be used to 
> have the engine skip certain constraints (e.g. skip warnings if we are 
> only interested in errors).
>
> Please clarify.
>
> Thanks,
> Holger
>