Re: Changes to prov:Dictionary

On Jun 6, 2012, at 5:05 PM, Luc Moreau wrote:

> Hi Tim,
> 
> See below
> 
> On 06/06/12 21:26, Timothy Lebo wrote:
>> 
>> Luc,
>> 
>> On Jun 6, 2012, at 2:36 PM, Luc Moreau wrote:
>> 
>>> 
>>> Hi all,
>>> 
>>> I dont understand this discussion.
>>> 
>>> See example 50
>>> http://dvcs.w3.org/hg/prov/raw-file/default/model/prov-dm.html#example_50
>> 
>> http://aquarius.tw.rpi.edu/prov-wg/prov-o#derivedByInsertionFrom
>> has a prov-o example very similar to #example_50.
> 
>> 
>>> We explicit list the contents of a dictionary after some insertion.
>> 
>> I agree with this example, what does it show that is wrong with prov-o?
>> 
>> 
> 
> I never claimed anything was wrong with prov-o.
> 
> Do you agree with the conclusions drawn in example 50, where the *exact* contents
> of the dictionaries are listed?

Ignoring the corners of the stage, yes.
i.e., yes.

> 
> 
>>> 
>>> Definition 38 in prov-constraints define membership
>>> http://dvcs.w3.org/hg/prov/raw-file/default/model/prov-constraints.html#membership-as-insertion
>> 
>> Great.
>> 
>>> 
>>> Disallowing complete membership seems to go against the definition of insertion.
>> 
>> How so?
>> "Insertion is a derivation that transforms a dictionary into another, by insertion of one or more key-entity pairs."
>> 
>> 
>> 
>>> Are you suggesting that we don't exactly know what is being inserted in a dictionary?
>> 
>> No, we know _exactly_ what is inserted into the dictionary. They inserted KeyValuePairs are listed right in the assertion.
>> 
> 
> 
>> 
>> What we're saying is that for any prov:Dictionary and for any N known members, there is no way to know if that prov:Dictionary only has N members, or if there are more.
> 
> So, if I understand you correctly, 
> 
> The following is fine:
> 
> memberOf(d1, {("k1", e1), ("k2", e2)} )  //d1 has the following pairs as members: ("k1", e1), ("k2", e2), and may contain others.
> 
> 
> The following should not be allowed.
> 
> memberOf(d2, {("k1", e1), ("k2", e2)}, true)  //d2 exactly has the following pairs as members: ("k1", e1), ("k2", e2), and does not contain any other.
> 
> The following is fine:
> 
> entity(d0,[prov:type='prov:EmptyDictionary'])
> derivedByInsertionFrom(d3, d0, {("k1", e1), ("k2", e2)})   //d3 exactly has the following pairs as members: ("k1", e1), ("k2", e2), and does not contain any other.

(where do we get "and no other" here? How are we "closing off" d3 so that something else couldn't be added to it by direct assertion?)

> But this latter expression is exactly 
> memberOf(d3, {("k1", e1), ("k2", e2)}, true) 
> as per constraint 38.

Yes, this is very nice.
So the complete flag is just promising that there exists a provenance trace back to nothing?



> 
> So, if we reject one expression and its interpretation,  shouldn't we also reject the other?

So, it's more like an "everything in here is accounted for" flag.
Which is half way to "complete"; the other half is knowing that "nothing else is in there".
I don't see how providing a provenance trace from nothing and ending up with N elements doesn't mean that there's N+1 elements.

What if I had missed a member-changing Activity in my trace?



> 
>> 
>> Membership is alive and healthy in this last PROV-O update announced in this thread.
> 
> Uuhh?

PROV-O is inconsistent with DM, and it's differences are intentional and an attempt to change the DM.
e.g., the deletion of prov:CompleteDictionary class in PROV-O proposes the removal of the "complete" flag in DM.
which is what we're discussing.

(BTW, I see enough good points that I'm ready to withdraw the charge, since those that asked for it aren't participating.)

-Tim


>> What is being proposed is that the class prov:CompleteDictionary (and DM's optional "complete" attribute)  be removed from PROV.
> 
> It still allows us to express complete membership, as per d3 above. 
> 
> I still don't understand. We can remove it, it doesn't seem to change anything in the model,
> it just  disallows some syntactic sugar.
> 
> Luc
> 
>> 
>> 
>> -Tim
>> 
>> 
>> 
>>> 
>>> 
>>> 
>>> Professor Luc Moreau
>>> Electronics and Computer Science
>>> University of Southampton 
>>> Southampton SO17 1BJ
>>> United Kingdom
>>> 
>>> On 6 Jun 2012, at 17:05, "Stephan Zednik" <zednis@rpi.edu> wrote:
>>> 
>>>> 
>>>> On Jun 6, 2012, at 10:57 AM, Timothy Lebo wrote:
>>>> 
>>>>> Stian,
>>>>> 
>>>>> On Jun 6, 2012, at 10:39 AM, Stian Soiland-Reyes wrote:
>>>>> 
>>>>>> Without EmptyCollection or CompleteMembership the
>>>>>> collections/dictionaries are of almost no worth to my use cases,
>>>>> 
>>>>> EmptyCollection remains in the latest PROV-O (so that is not an issue).
>>>>> 
>>>>> It was CompleteMembership that got the ax (this is the topic at hand).
>>>>> 
>>>>> Regarding your use cases, I think it's important to cite Graham's points about uses cases for standards:
>>>>> http://www.w3.org/mid/4FCEFCB0.4090100@zoo.ox.ac.uk
>>>> 
>>>> +1 to the relevance of Graham's point about scope creep and system use cases vs. coverage/scope of a standard.
>>>> 
>>>>> 
>>>>> 
>>>>> 
>>>>>> as
>>>>>> all I can say then is that "some of the members are X, Y and Z" - but
>>>>>> there might also be A, B and C.
>>>> 
>>>> Are you specifically worried about the possibility that other members may be asserted at a later time by someone else?  If this is an issue than perhaps you could use a system-specific extension of prov:Collection which utilizes a terminated ordered list.
>>>> 
>>>> I must reiterate my agreement with Graham's point above that this need from this use case should not become a requirement for all collections defined in the standard.
>>>> 
>>>>>> In Taverna workflows, all collections
>>>>>> are closed (unless you export provenance before a workflow has
>>>>>> finished). It is important to know that ALL these genes - and no other
>>>>>> genes - came back. Just saying "some of these came back" is of less
>>>>>> value.
>>>>> 
>>>>> Would this use case be handled if Taverna instead leveraged the "additional attributes" that DM already provides?
>>>>> 
>>>>> memberOf(id; c, {(key_1, e_1), ..., (key_n, e_n)}, cplt, attrs)
>>>>> 
>>>>> perhaps a property taverna:isComplete or class taverna:CompleteDictionary ?
>>>> 
>>>> Even if this attribute was added to the prov or an extension of prov, it does not enforce the closed-world membership that Stian would like to have.
>>>> 
>>>> No attribute or class specialization will resolve the issue of trying to enforce CWA in RDF.
>>>> 
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> I understand that in RDF if we don't use rdf:List, then statements of
>>>>>> such completeness are still fairly vague as the lists are not
>>>>>> terminated and additional tuples could be adding
>>>>>> members/insertions/removals.
>>>>> 
>>>>> If this can't be handled soundly and properly in PROV-O and OWA, then I don't think we should try (or, fake it).
>>>> 
>>>> +1
>>>> 
>>>> I think in general the idea of 'completeness' is incompatible with OWA and should not be addressed in PROV-O.
>>>> 
>>>> --Stephan
>>>> 
>>>>> 
>>>>>> 
>>>>>> However when I make a provenance export of a workflow run, I would
>>>>>> want to also say something like "These are all the workflow processes
>>>>>> that ran, and these are all the entities that were created".
>>>>> 
>>>>> 
>>>>> 
>>>>>> But
>>>>>> perhaps a more general completeness-claim for an account/bundle is out
>>>>>> of scope for PROV.
>>>>> 
>>>>> That seems to be the predominant perspective, as people have indicated in various email threads and tracker issues.
>>>>> With the use of a custom attribute and type ( taverna:isComplete or class taverna:CompleteDictionary ), can you accept removing the special optional parameter on DM's memberOf?
>>>>> 
>>>>>> 
>>>>>> 
>>>>>> However, I still don't undertstand what is the problem with saying
>>>>>> something is an empty collection.
>>>>> 
>>>>> Not an issue. EmptyDictionary is still in there :-)
>>>>> 
>>>>> Thanks!
>>>>> Tim
>>>>> 
>>>>>> 
>>>>>> On Wed, Jun 6, 2012 at 2:49 PM, Luc Moreau <L.Moreau@ecs.soton.ac.uk> wrote:
>>>>>>> 
>>>>>>> Hi Tim,
>>>>>>> 
>>>>>>> It's specifically your last point. Being to express whether membership was complete
>>>>>>> was a request from Stian and Paolo I believe.
>>>>>>> 
>>>>>>> Luc
>>>>>>> 
>>>>>>> 
>>>>>>> On 06/06/2012 02:31 PM, Timothy Lebo wrote:
>>>>>>> 
>>>>>>> Luc,
>>>>>>> 
>>>>>>> On Jun 6, 2012, at 12:48 AM, Luc Moreau wrote:
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> On 5 Jun 2012, at 23:18, "Timothy Lebo" <lebot@rpi.edu> wrote:
>>>>>>> 
>>>>>>> prov-wg (and prov-dm editors),
>>>>>>> 
>>>>>>> I've reviewed all of the materials (that I can find) regarding collective concerns about prov:Dictionary, and
>>>>>>> have committed changes to the latest PROV-O owl and html to address those concerns:
>>>>>>> 
>>>>>>> * https://dvcs.w3.org/hg/prov/raw-file/default/ontology/Overview.html
>>>>>>> * http://dvcs.w3.org/hg/prov/raw-file/default/ontology/ProvenanceOntology.owl
>>>>>>> 
>>>>>>> The changes are summarized here:
>>>>>>> 
>>>>>>> http://www.w3.org/2011/prov/wiki/index.php?title=Eg-34-us-supreme-court-membership&oldid=7905#PROV-O_changes_made.2C_inspired_by_this_example
>>>>>>> 
>>>>>>> and repeated here:
>>>>>>> 
>>>>>>>  Added class prov:Collection, as subclass of Entity
>>>>>>> Added property prov:hadMember domain prov:Collection range prov:Entity.
>>>>>>> 
>>>>>>> This supports both generic "simple set" prov:Collection and prov:Dictionary.
>>>>>>> 
>>>>>>> Made KeyValuePair a subclass of Entity
>>>>>>> 
>>>>>>> this follows from Set Collection :c prov:hadMember :my_member and the definition of Collection "A collection is an entity that has some members. The members are themselves entities").
>>>>>>> 
>>>>>>> Renamed prov:membership to prov:qualifiedMembership to follow qualification pattern naming.
>>>>>>> prov:Membership became subclass of prov:EntityInvolvement (though, it could become subclass of prov:KeyValuePairInvolvement, itself a subclass of prov:EntityInvolvement. But we'll try to simplify and reuse prov:entity)
>>>>>>> prov:member renamed to prov:pair and became a subproperty of prov:involvee
>>>>>>> Added property chain (qualifiedMembership o prov:pair) rdfs:subClassOf prov:hadMember
>>>>>>> Added prov:removed domain prov:Removal range prov:KeyValuePair
>>>>>>> Removed prov:CompleteDictionary from DM and PROV-O.
>>>>>>> 
>>>>>>> Why?
>>>>>>> Luc
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> What in particular would you like to discuss.
>>>>>>> As I said, this reflects a response to many concerns that have been raised by many people in many forms.
>>>>>>> In an effort to maintain focus and to make progress, I recommend that these points, the latest prov-dm, and the latest prov-o update serve as the basis for these discussions.
>>>>>>> 
>>>>>>> -Tim
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> You'll notice the prov-o modeling of Dictionaries is not consistent with latest prov-dm.
>>>>>>> 
>>>>>>> The prov-o team would like to ask the prov-dm editors to reconsider how collections and dictionaries are defined, so that they reflect the latest prov-o modeling of the PROV concepts.
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Tim Lebo
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> cc tracker ISSUE-374 ISSUE-391
>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>>>> --
>>>>>>> Professor Luc Moreau
>>>>>>> Electronics and Computer Science   tel:   +44 23 8059 4487
>>>>>>> University of Southampton          fax:   +44 23 8059 2865
>>>>>>> Southampton SO17 1BJ               email: l.moreau@ecs.soton.ac.uk
>>>>>>> United Kingdom                     http://www.ecs.soton.ac.uk/~lavm
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> 
>>>>>> --
>>>>>> Stian Soiland-Reyes, myGrid team
>>>>>> School of Computer Science
>>>>>> The University of Manchester
>>>>>> 
>>>>>> 
>>>>> 
>>>> 
>>