Re: HTTP requests, databases and how to transmit SHACL validation reports

On 04/06/2020 21:54, "Håvard M. Ottestad" wrote:
> Hi,
> 
> We need to detect if the returned data is a SHACL validation report so that we can throw a ShaclValidationException in our RDF4J Java client. Parsing the data would consume the stream, which means we can’t as easily stream the data through to the user/developer, parsing the data could also not fit in memory.

My point is that registration is for the web, not just specific cases.
Is it necessary to knoe its a validation report from the header. That 
description is a desirable in a case of passthrough.

e.g. Validation reports aren't always large.

Maybe we should have a "limit" on errors.

Maybe "distinct" violation reports, because volume may means the same 
constraint failure many times.

(HTTP is pretty poor about very large request or responses - it does 
rather assume they are not the dominate case.)

There are other ways to achieve the benefits of streaming/passthru even 
if streaming is a requirement. Yes, implementation is harder - but it is 
localised, not putting into more complexity into every systems HTTP 
interaction.


It is possible to optimize in an RDF stream. (Plain) JSON and XML 
processors optimize for fields in the "right" order and cope/degrade 
when they aren't. They preserve the "whole file" nature of JSON and XML 
because the doc isn't valid unless it ends properly.

<Insert usual issues if the stream is truncated>

One way is optimize for the triples where the stream starts "rdf:type 
sh:ValidationReport" - sniff the stream.

So an alternative approach here is writing a "best practice" note on how 
to handle 409 responses. "Optimize the best, cope with the rest."

> I don’t see how a SHACL validation violation would produce anything other than a SHACL validation report.
> 
> My initial thought was a custom header and standard mime type.

Vladimir has mentioned profiles:

https://www.w3.org/TR/dx-prof-conneg/

which look like a progressive approach.

So I see there are choices to consider before making a de-facto 
registration (a CG can't formally register a MIME type). Any design is a 
tradeoff.

     Andy

> 
> Håvard
> 
>> On 4 Jun 2020, at 22:22, Andy Seaborne <andy@apache.org> wrote:
>>
>> On reflection, I don't understand the need for a specific MIME type for the 409 response.
>>
>> What is the use case that requires adding the meaning to the MIME type, as compared to, say, parse as RDF and inspect the data?
>>
>> What about other content, more than the validation report?
>>
>> And IF there is a requirement to put it in advance of the body, is this the right mechanism?
>>
>> e.g.
>> https://www.w3.org/TR/dx-prof-conneg/
>>
>>     Andy
>>
>>> On 22/03/2020 18:51, Håvard Ottestad wrote:
>>> Thanks Andy. I really appreciate your input :)
>>> Håvard
>>>>> On 22 Mar 2020, at 19:25, Andy Seaborne <andy@apache.org> wrote:
>>>>
>>>> Good question, good timing.
>>>>
>>>> On 22/03/2020 16:23, Håvard Ottestad wrote:
>>>>> Hi,
>>>>> I’m a committer over at Eclipse RDF4J and am currently looking at how we can extend our RDF4J Server to respond to the client with the SHACL validation report if the user's request could not be processed due to a violation.
>>>>> An example:
>>>>> We have a database of users with some SHACL rules that among other things specify that a user needs to have exactly one password.
>>>>> The client tries to add a user, but the user didn’t specify a password, so the request gets rejected by the database. This should naturally have been detected on the client side first - but you know… lazy developers and the like :P
>>>>> Anyway, since this is happening over HTTP we need to send an appropriate response to the client so that the client can tell the user to provide a password. This response should contain the SHACL validation report, we don’t want to reinvent the wheel.
>>>>> For the actual HTTP response we have a few things to consider:
>>>>>   - Status and status line. Currently considering 409 - Conflict
>>>>
>>>> Loos good.
>>>>
>>>> While some 4xx and 5xx are errors specific to the HTTP protocol (e.g. 406 "Not Acceptable"), 409 does say the current state of the resource conflicts with the request which seems appropriate here.
>>>>
>>>>>   - Content-type header. Currently considering this to be "application/n-quads" to reflect the serialisation of the SHACL validation report
>>>>
>>>> Yes - but to me application/shacl-validation-report+n-quads is better.
>>>>
>>>> Realistically, the "x+y" thing isn't well supported by client-side toolkits but mist do expose the whole Content-type of the response.
>>>>
>>>>>   - Custom header. Currently considering "Eclipse-RDF4J-Exception: ShaclSailValidationException” to tell the client that this response is due to a ShaclSailValidationException
>>>>
>>>> Custom headers can be inconvenient depending on the client-side HTTP library being used.  To be useful the app is going to look in the body so I'm not sure this adds much over the "Content-type" option.
>>>>
>>>>> We have other options too:
>>>>>   - "Content-type: application/n-quads+shacl-validation-report” similar to how json-ld is handled
>>>>
>>>> Yes, though the other way round:
>>>>
>>>> application/shacl-validation-report+n-quads
>>>>
>>>> c.f
>>>>
>>>> rdf+xml
>>>> sparql-results+json
>>>> ld+json
>>>>
>>>> and
>>>> application/shacl-validation-report+turtle
>>>>
>>>> if you must:
>>>> text/shacl-validation-report+turtle
>>>>
>>>> (always did think text/turtle was a bit strange)
>>>>
>>>>>   - Custom status: 409 - SHACL violation
>>>>
>>>> HTTP/2 and HTTP/3 don't carry the message part of HTTP/1.1 status line.
>>>> https://tools.ietf.org/html/rfc7540#section-8.1.2.4
>>>>
>>>>>   - Magic numbers train of thought with the first line of the response being something like “SHACL validation report”.
>>>>
>>>> That would be a new MIME type?
>>>>
>>>> I prefer application/shacl-validation-report+turtle
>>>>
>>>>> What I’m wondering is: How are other databases solving this particular problem?
>>>>> And: Did the SHACL Working Group discuss this issue? Maybe in light of the SPARQL protocol?
>>>>> Best regards,
>>>>> Håvard M. Ottestad
>>>>
>>

Received on Thursday, 4 June 2020 23:24:47 UTC