RE: Q about reconciliation query batch size

Thanks, all, for your replies.  I like the idea of advertising the limit to clients and/or a standard error pattern.

I can appreciate intentionally remaining silent on these types of issues, so as not to overly constrain an implementer, but I also worry about a implementers each finding their own solutions, which could result in a variation that clients must navigate per service.  Advertising limits helps to establish an understanding of what the service can or is willing to do and what the client can expect.  If the client breaches that understanding, the client had access to the rules prior.

I thought Tom brought up some interesting questions/scenarios, especially with respect to simultaneous parallel requests from the same client.  Perhaps the spec remains silent on these points for now?

Yours,
Kevin


From: Antonin Delpeuch <antonin@delpeuch.eu>
Sent: Monday, June 15, 2020 4:35 PM
To: David Newbury <DNewbury@getty.edu>; public-reconciliation@w3.org
Subject: Re: Q about reconciliation query batch size


Yes, my earlier message was unclear (or even grossly misleading): this should totally be configurable by the service, for instance in the manifest.

Antonin
On 15/06/2020 22:31, David Newbury wrote:
I would lean towards either an advertisable value or a standard error pattern.  Different systems will have radically different load patterns, and we probably don’t want to accidently prevent people from building systems in ways that make sense.

— David

David Newbury, Enterprise Software Architect  |  The J. Paul Getty Trust |  (310) 440 6116  |  getty.edu<http://www.getty.edu/>

[../../Getty_Logo_EmailSig.png]

From: Antonin Delpeuch <antonin@delpeuch.eu><mailto:antonin@delpeuch.eu>
Date: Saturday, June 13, 2020 at 11:56 PM
To: "public-reconciliation@w3.org"<mailto:public-reconciliation@w3.org> <public-reconciliation@w3.org><mailto:public-reconciliation@w3.org>
Subject: Re: Q about reconciliation query batch size
Resent-From: <public-reconciliation@w3.org><mailto:public-reconciliation@w3.org>
Resent-Date: Saturday, June 13, 2020 at 11:55 PM



On 11/06/2020 17:58, Ford, Kevin wrote:
 I’m familiar in an academic sense with OpenRefine, but not whether it might control the size of query batches to ensure a provider is not overwhelmed.  That said, if this work is to become a more generic way to provide reconciliation or suggest services to be used by software other than OpenRefine, then it still seems this should be an advertiseable/controllable value since one cannot always count on the client being responsible.

I think it would make sense to improve the specs on this: for instance, to specify a maximum batch size beyond which the service will refuse to respond to requests.

Antonin


CAUTION: This email originated from outside of the Getty. Do not click links or open attachments unless you verify the sender and know the content is safe.