Re: Why not adopt ShEx? (was Re: Enough already)

Hi Arnaud,

I don’t think we need to worry that the implementation threshold will not be met. A number of projects are starting up. I also hear that several organizations are planning to build of the Jena implementation, which to me makes sense.

Implementers will act if there is customer interest and pressure. This is true for TopQuadrant and I am sure for other vendors as well. When we advocate for something it is almost never because it is our idea and we think it is good. Rather, it is because of the demand from a significant number of customers who, if they don’t get it, will go elsewhere. This is why I believe the user interest and enthusiasm are actually the most important factors. They will drive the market.

On the topic of meeting the deadlines, I am all for reducing the scope and using all other best practices for getting this project done on time. However, I didn’t see how this particular move would have helped with it as of the 23 open issues, only 1 is about extensions. The other 22 are all about core. Further, removing extensions would also have a cost of re-writing the spec. So, the net effect on the effort and time would have likely been negative or, at the very best, neutral.

I think setting intermediate short term deadlines and focusing the team on meeting them, time boxing discussions so that they don’t overrun, enforcing more effective management of the precious meeting time and other standard project management techniques are key and, at this point, the best bets for timely completion. As someone just said in an e-mail, “don’t let perfect to become the enemy of the good". These days "publish and evolve” approach has replaced “complete and perfect”. This is much more pragmatic and delivers better results for the entire community - in both the short and the long term.

Regards,


Irene
> On Dec 13, 2016, at 3:55 AM, Arnaud Le Hors <lehors@us.ibm.com> wrote:
> 
> Hi Irene,
> 
> You're right that it is always interesting to hear from users. But for a technology to become part of a W3C Recommendation it needs to have at least two independent implementations. To date, while I know of several implementations of Core, I've only heard of one implementation of the SPARQL extension: TopQuadrant's. So my questions are motivated by the need to find out if there are any others. Unfortunately it appears that so far most of the people we've heard from are TopQuadrant customers rather than implementers.
> 
> Besides, when it comes to judging of the adequacy of a draft to become a W3C Recommendation the opinion of implementers is paramount because if they find the specification too imprecise to be able to implement it with confidence that their implementation will behave like others we have little chance of reaching interoperability across products, which is what standards are all about.
> 
> It seems that people are being rallied to make the point that the SPARQL extension is important to them. I don't know whether anybody questions that but I certainly don't. My proposal to separate it from Core was merely motived by reducing the scope of work. I understand this is not to everyone's liking but removing features to reduce the scope of work is common practice when facing an imminent deadline and it was suggested by W3M. TopQuadrant will apparently object to it. I heard you, let's move on.
> --
> Arnaud  Le Hors - Senior Technical Staff Member, Open Web & Blockchain Technologies - IBM Cloud
> 
> 
> 
> 
> From:        Irene Polikoff <irene@topquadrant.com>
> To:        Arnaud Le Hors/Cupertino/IBM@IBMUS
> Cc:        Dean Allemang <dallemang@workingontologist.com>, "Eric Prud'hommeaux" <eric@w3.org>, public-rdf-shapes@w3.org
> Date:        12/13/2016 12:26 AM
> Subject:        Re: Why not adopt ShEx? (was Re: Enough already)
> 
> 
> 
> Arnaud,
> 
> I would not people to think that your position is that unless they are an implementer of a SHACL engine, their input to the working group is not of value.
> 
> Your responses to the recent posts on this forum come across as tightly connecting the user comments and the implementations of SHACL. I think many folks that have commented are the current and potential future users of SHACL who are providing input on the features/requirements that are important to them. That is, users who would create and use SHACL shapes as opposed to those that would develop their own SHACL engines. 
> 
> Your questions might be interpreted as if you expect every user have to write their own engine. I believe user input to be very important and informative to the working group. I am sure that you would agree.
> 
> Regards,
> 
> Irene
> 
> On Dec 12, 2016, at 3:41 PM, Arnaud Le Hors <lehors@us.ibm.com <mailto:lehors@us.ibm.com>> wrote:
> 
> Hi Dean,
> So what do you use for this work? Did you implement SHACL Full?
> Thanks.
> --
> Arnaud  Le Hors - Senior Technical Staff Member, Open Web & Blockchain Technologies - IBM Cloud
> 
> 
> 
> 
> From:        Dean Allemang <dallemang@workingontologist.com <mailto:dallemang@workingontologist.com>>
> To:        "Eric Prud'hommeaux" <eric@w3.org <mailto:eric@w3.org>>
> Cc:        public-rdf-shapes@w3.org <mailto:public-rdf-shapes@w3.org>
> Date:        12/12/2016 09:05 PM
> Subject:        Re: Why not adopt ShEx? (was Re: Enough already)
> Sent by:        deanallemang@gmail.com <mailto:deanallemang@gmail.com>
> 
> 
> 
> There are three validation operations I can imagine wanting to perform
> on an engine that supports SPARQL:
> 
> This is a lovely idea, but it isn't realistic from the beginning.  We are in the position of having a vendor work with us, with their current technology stack.  We can't expect them (and we don't have time to wait; once a PoC is approved, it is usually due yesterday) to adapt their stack to something new.  Of course, once we have a Recommendation in place, it is a lot easier for us to request compliance to it.  I am interested in a smooth slope to adoption.  A SPARQL based constraint system is easy to adopt in the current environment.  As adoption of the standard progresses, adherence to acceptance criteria will be a good idea (and we would be interested in cooperating in developing those)
> 
>  3 Extend the SPARQL engine to support SHACL Full's extensibility
>    mechanism.
> 
> I find this criterion to be particularly interesting.  As a standards organization ourselves, I don't think we are in a position to make use of this (are we going to publish our own extensions to SHACL? I don't think so).  But from an implementation point of view, this is really awesome.   I don't know that we would need to have candidate POC vendors pass this one.
> 
> Dean
> 
> 
> 
> 
> 
> On Mon, Dec 12, 2016 at 2:48 PM, Eric Prud'hommeaux <eric@w3.org <mailto:eric@w3.org>> wrote:
> * Dean Allemang <dallemang@workingontologist.com <mailto:dallemang@workingontologist.com>> [2016-12-12 10:33-0500]
> > >
> > > Sorry, but I see zero advantages of ShEx over SPIN/SPARQL.
> > >
> > > Why would I want to lock my software into a new non-standard syntax with
> > > close to none adoption, when I can simply use the query engine to validate
> > > constraints?
> > >
> > >
> > >
> >
> > I couldn't agree more.  In FIBO,  we have been looking for a constraint
> > language to help us make definitions that go beyond the capabilities of
> > OWL.  I presented these at the inaugural meeting of the SHAPES group a
> > couple years ago.  It is easy to specify them in SPARQL, and we have done
> > so (and I did the same in SHACL, now that there is a write-up of how it
> > works).
> >
> > When we move from our conceptual ontology to something operational for a
> > Proof of Concept, some vendor is always  involved.  That vendor (different
> > one for each PoC) always has an RDF store somewhere in their stack.  They
> > can always consume OWL (though often through a rule engine interpretations
> > via OWL2RL).  For rules/constraints that go beyond OWL, we have to work out
> > some way to give them the rules.   SWRL?  Some of them can manage that.
> > RIF?  Everyone knows what it is, but few can handle it out of the box.
> > Other rule systems have varying degrees of uptake.
> >
> > But one thing all the triple stores can manage is SPARQL.  "How about if I
> > give you the constraints in SPARQL?"  the answer is always, "Oh, sure, that
> > works".  Because they are all triple stores, and they already do it.
> 
> There are three validation operations I can imagine wanting to perform
> on an engine that supports SPARQL:
> 
>  1 Validate ShEx or SHACL core over a the SPARQL protocol.
> 
>  2 Validate ShEx or SHACL core over a graph API.
> 
>  3 Extend the SPARQL engine to support SHACL Full's extensibility
>    mechanism.
> 
> For 1 and 2, I think ShEx and SHACL are about equal. Peter's
> implementation of SHACL Core used a mixture of graph API and SPARQL
> but could certainly have been implemented just in terms of the
> ubiquitous triplesMatching API. The ShEx demo compiles ShEx 1 to
> SPARQL queries to run over the SPARQL protocol but of course that
> didn't support features like told bnodes (identifying a bnode by
> label).
> 
> I think SPARQL becomes relevent when you want to build a SHACL Full
> (or SPIN) engine. You would have to implement a full SPARQL engine
> *and then* build the node/shape iterators, templating system, and
> recursion control that are required for SHACL Full.
> 
> 
> > This doesn't mean that we have to do this in SPARQL, but it does mean that
> > if we have that option, we shortcut a lot of work to get to our Proofs of
> > Concept.
> >
> > In the end, I'm just re-iterating what Martynas has said much more
> > succinctly, but in the context of a whole industry effort (FIBO) and a
> > selection of vendors who want to work with us.
> >
> >
> >
> > Dean
> 
> --
> -ericP
> 
> office: +1.617.599.3509 <tel:%2B1.617.599.3509>
> mobile: +33.6.80.80.35.59 <tel:%2B33.6.80.80.35.59>
> 
> (eric@w3.org <mailto:eric@w3.org>)
> Feel free to forward this message to any list for any purpose other than
> email address distribution.
> 
> There are subtle nuances encoded in font variation and clever layout
> which can only be seen by printing this message on high-clay paper.
> 
> 
> 
> 
> 
>