Re: Changes to prov:Dictionary

Hi Tim,

Response below.

On 06/07/2012 01:16 PM, Timothy Lebo wrote:
>
> On Jun 6, 2012, at 5:05 PM, Luc Moreau wrote:
>
>> Hi Tim,
>>
>> See below
>>
>> On 06/06/12 21:26, Timothy Lebo wrote:
>>> Luc,
>>>
>>> On Jun 6, 2012, at 2:36 PM, Luc Moreau wrote:
>>>
>>>>
>>>> Hi all,
>>>>
>>>> I dont understand this discussion.
>>>>
>>>> See example 50
>>>> http://dvcs.w3.org/hg/prov/raw-file/default/model/prov-dm.html#example_50
>>>
>>> http://aquarius.tw.rpi.edu/prov-wg/prov-o#derivedByInsertionFrom
>>> has a prov-o example very similar to #example_50.
>>
>>>
>>>> We explicit list the contents of a dictionary after some insertion.
>>>
>>> I agree with this example, what does it show that is wrong with prov-o?
>>>
>>>
>>
>> I never claimed anything was wrong with prov-o.
>>
>> Do you agree with the conclusions drawn in example 50, where the 
>> *exact* contents
>> of the dictionaries are listed?
>
> Ignoring the corners of the stage, yes.
> i.e., yes.
>

ok
>>
>>
>>>>
>>>> Definition 38 in prov-constraints define membership
>>>> http://dvcs.w3.org/hg/prov/raw-file/default/model/prov-constraints.html#membership-as-insertion 
>>>>
>>>
>>> Great.
>>>
>>>>
>>>> Disallowing complete membership seems to go against the definition 
>>>> of insertion.
>>>
>>> How so?
>>> "Insertion is a derivation that transforms a dictionary into 
>>> another, by insertion of one or more key-entity pairs."
>>>
>>>
>>>
>>>> Are you suggesting that we don't exactly know what is being 
>>>> inserted in a dictionary?
>>>
>>> No, we know _exactly_ what is inserted into the dictionary. They 
>>> inserted KeyValuePairs are listed right in the assertion.
>>>
>>
>>
>>>
>>> What we're saying is that for any prov:Dictionary and for any N 
>>> known members, there is no way to know if that prov:Dictionary only 
>>> has N members, or if there are more.
>>
>> So, if I understand you correctly,
>>
>> The following is fine:
>>
>> memberOf(d1, {("k1", e1), ("k2", e2)} )  //d1 has the following pairs 
>> as members: ("k1", e1), ("k2", e2), and may contain others.
>>
>>
>> The following should not be allowed.
>>
>> memberOf(d2, {("k1", e1), ("k2", e2)}, true)  //d2 exactly has the 
>> following pairs as members: ("k1", e1), ("k2", e2), and does not 
>> contain any other.
>>
>> The following is fine:
>>
>> entity(d0,[prov:type='prov:EmptyDictionary'])
>> derivedByInsertionFrom(d3, d0, {("k1", e1), ("k2", e2)})   //d3 
>> exactly has the following pairs as members: ("k1", e1), ("k2", e2), 
>> and does not contain any other.
>
> (where do we get "and no other" here? How are we "closing off" d3 so 
> that something else couldn't be added to it by direct assertion?)
>

There is only one permitted predecessor to a dictionary:
http://dvcs.w3.org/hg/prov/raw-file/default/model/prov-constraints.html#dictionary-unique-derivation

>> But this latter expression is exactly
>> memberOf(d3, {("k1", e1), ("k2", e2)}, true)
>> as per constraint 38.
>
> Yes, this is very nice.
> So the complete flag is just promising that there exists a provenance 
> trace back to nothing?
>
>

yes, back to the empty dictionary.

>
>>
>> So, if we reject one expression and its interpretation,  shouldn't we 
>> also reject the other?
>
> So, it's more like an "everything in here is accounted for" flag.
> Which is half way to "complete"; the other half is knowing that 
> "nothing else is in there".
> I don't see how providing a provenance trace from nothing and ending 
> up with N elements doesn't mean that there's N+1 elements.
>
> What if I had missed a member-changing Activity in my trace?
>
>

if you had
derivedByInsertionFrom(d2,d1, {("k3", e3), ("k4", e4)})
derivedByInsertionFrom(d1,d0, {("k1", e1), ("k2", e2)})

and you realize that (k5,e5) was also introduced to the dictionary when 
it was in state d1,
you can express:

derivedByInsertionFrom(d3,d1, {("k3", e3), ("k4", e4), ("k5,d5)})


This is permitted by
http://dvcs.w3.org/hg/prov/raw-file/default/model/prov-constraints.html#dictionary-branching

At some point you thought the state of the dictionary was d2, at another,
you thought it was d3.

>
>>
>>>
>>> Membership is alive and healthy in this last PROV-O update announced 
>>> in this thread.
>>
>> Uuhh?
>
> PROV-O is inconsistent with DM, and it's differences are intentional 
> and an attempt to change the DM.
> e.g., the deletion of prov:CompleteDictionary class in PROV-O proposes 
> the removal of the "complete" flag in DM.
> which is what we're discussing.
>
> (BTW, I see enough good points that I'm ready to withdraw the charge, 
> since those that asked for it aren't participating.)

I don't regard these as "charge".  These are valid points.
You may want to look at my summary, which Paul added to today's agenda.

Luc

>
> -Tim
>
>
>>> What is being proposed is that the class prov:CompleteDictionary 
>>> (and DM's optional "complete" attribute)  be removed from PROV.
>>
>> It still allows us to express complete membership, as per d3 above.
>>
>> I still don't understand. We can remove it, it doesn't seem to change 
>> anything in the model,
>> it just  disallows some syntactic sugar.
>>
>> Luc
>>
>>>
>>>
>>> -Tim
>>>
>>>
>>>
>>>>
>>>>
>>>>
>>>> Professor Luc Moreau
>>>> Electronics and Computer Science
>>>> University of Southampton
>>>> Southampton SO17 1BJ
>>>> United Kingdom
>>>>
>>>> On 6 Jun 2012, at 17:05, "Stephan Zednik" <zednis@rpi.edu 
>>>> <mailto:zednis@rpi.edu>> wrote:
>>>>
>>>>>
>>>>> On Jun 6, 2012, at 10:57 AM, Timothy Lebo wrote:
>>>>>
>>>>>> Stian,
>>>>>>
>>>>>> On Jun 6, 2012, at 10:39 AM, Stian Soiland-Reyes wrote:
>>>>>>
>>>>>>> Without EmptyCollection or CompleteMembership the
>>>>>>> collections/dictionaries are of almost no worth to my use cases,
>>>>>>
>>>>>> EmptyCollection remains in the latest PROV-O (so that is not an 
>>>>>> issue).
>>>>>>
>>>>>> It was CompleteMembership that got the ax (this is the topic at 
>>>>>> hand).
>>>>>>
>>>>>> Regarding your use cases, I think it's important to cite Graham's 
>>>>>> points about uses cases for standards:
>>>>>> http://www.w3.org/mid/4FCEFCB0.4090100@zoo.ox.ac.uk
>>>>>
>>>>> +1 to the relevance of Graham's point about scope creep and system 
>>>>> use cases vs. coverage/scope of a standard.
>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>> as
>>>>>>> all I can say then is that "some of the members are X, Y and Z" 
>>>>>>> - but
>>>>>>> there might also be A, B and C.
>>>>>
>>>>> Are you specifically worried about the possibility that other 
>>>>> members may be asserted at a later time by someone else?  If this 
>>>>> is an issue than perhaps you could use a system-specific extension 
>>>>> of prov:Collection which utilizes a terminated ordered list.
>>>>>
>>>>> I must reiterate my agreement with Graham's point above that this 
>>>>> need from this use case should not become a requirement for all 
>>>>> collections defined in the standard.
>>>>>
>>>>>>> In Taverna workflows, all collections
>>>>>>> are closed (unless you export provenance before a workflow has
>>>>>>> finished). It is important to know that ALL these genes - and no 
>>>>>>> other
>>>>>>> genes - came back. Just saying "some of these came back" is of less
>>>>>>> value.
>>>>>>
>>>>>> Would this use case be handled if Taverna instead leveraged the 
>>>>>> "additional attributes" that DM already provides?
>>>>>>
>>>>>> memberOf(id; c, {(key_1, e_1), ..., (key_n, e_n)}, cplt, attrs)
>>>>>>
>>>>>> perhaps a property taverna:isComplete or class 
>>>>>> taverna:CompleteDictionary ?
>>>>>
>>>>> Even if this attribute was added to the prov or an extension of 
>>>>> prov, it does not enforce the closed-world membership that Stian 
>>>>> would like to have.
>>>>>
>>>>> No attribute or class specialization will resolve the issue of 
>>>>> trying to enforce CWA in RDF.
>>>>>
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> I understand that in RDF if we don't use rdf:List, then 
>>>>>>> statements of
>>>>>>> such completeness are still fairly vague as the lists are not
>>>>>>> terminated and additional tuples could be adding
>>>>>>> members/insertions/removals.
>>>>>>
>>>>>> If this can't be handled soundly and properly in PROV-O and OWA, 
>>>>>> then I don't think we should try (or, fake it).
>>>>>
>>>>> +1
>>>>>
>>>>> I think in general the idea of 'completeness' is incompatible with 
>>>>> OWA and should not be addressed in PROV-O.
>>>>>
>>>>> --Stephan
>>>>>
>>>>>>
>>>>>>>
>>>>>>> However when I make a provenance export of a workflow run, I would
>>>>>>> want to also say something like "These are all the workflow 
>>>>>>> processes
>>>>>>> that ran, and these are all the entities that were created".
>>>>>>
>>>>>>
>>>>>>
>>>>>>> But
>>>>>>> perhaps a more general completeness-claim for an account/bundle 
>>>>>>> is out
>>>>>>> of scope for PROV.
>>>>>>
>>>>>> That seems to be the predominant perspective, as people have 
>>>>>> indicated in various email threads and tracker issues.
>>>>>> With the use of a custom attribute and type ( taverna:isComplete 
>>>>>> or class taverna:CompleteDictionary ), can you accept removing 
>>>>>> the special optional parameter on DM's memberOf?
>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> However, I still don't undertstand what is the problem with saying
>>>>>>> something is an empty collection.
>>>>>>
>>>>>> Not an issue. EmptyDictionary is still in there :-)
>>>>>>
>>>>>> Thanks!
>>>>>> Tim
>>>>>>
>>>>>>>
>>>>>>> On Wed, Jun 6, 2012 at 2:49 PM, Luc Moreau 
>>>>>>> <L.Moreau@ecs.soton.ac.uk <mailto:L.Moreau@ecs.soton.ac.uk>> wrote:
>>>>>>>>
>>>>>>>> Hi Tim,
>>>>>>>>
>>>>>>>> It's specifically your last point. Being to express whether 
>>>>>>>> membership was complete
>>>>>>>> was a request from Stian and Paolo I believe.
>>>>>>>>
>>>>>>>> Luc
>>>>>>>>
>>>>>>>>
>>>>>>>> On 06/06/2012 02:31 PM, Timothy Lebo wrote:
>>>>>>>>
>>>>>>>> Luc,
>>>>>>>>
>>>>>>>> On Jun 6, 2012, at 12:48 AM, Luc Moreau wrote:
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> On 5 Jun 2012, at 23:18, "Timothy Lebo" <lebot@rpi.edu 
>>>>>>>> <mailto:lebot@rpi.edu>> wrote:
>>>>>>>>
>>>>>>>> prov-wg (and prov-dm editors),
>>>>>>>>
>>>>>>>> I've reviewed all of the materials (that I can find) regarding 
>>>>>>>> collective concerns about prov:Dictionary, and
>>>>>>>> have committed changes to the latest PROV-O owl and html to 
>>>>>>>> address those concerns:
>>>>>>>>
>>>>>>>> * 
>>>>>>>> https://dvcs.w3.org/hg/prov/raw-file/default/ontology/Overview.html
>>>>>>>> * 
>>>>>>>> http://dvcs.w3.org/hg/prov/raw-file/default/ontology/ProvenanceOntology.owl
>>>>>>>>
>>>>>>>> The changes are summarized here:
>>>>>>>>
>>>>>>>> http://www.w3.org/2011/prov/wiki/index.php?title=Eg-34-us-supreme-court-membership&oldid=7905#PROV-O_changes_made.2C_inspired_by_this_example 
>>>>>>>> <http://www.w3.org/2011/prov/wiki/index.php?title=Eg-34-us-supreme-court-membership&oldid=7905#PROV-O_changes_made.2C_inspired_by_this_example>
>>>>>>>>
>>>>>>>> and repeated here:
>>>>>>>>
>>>>>>>>  Added class prov:Collection, as subclass of Entity
>>>>>>>> Added property prov:hadMember domain prov:Collection range 
>>>>>>>> prov:Entity.
>>>>>>>>
>>>>>>>> This supports both generic "simple set" prov:Collection and 
>>>>>>>> prov:Dictionary.
>>>>>>>>
>>>>>>>> Made KeyValuePair a subclass of Entity
>>>>>>>>
>>>>>>>> this follows from Set Collection :c prov:hadMember :my_member 
>>>>>>>> and the definition of Collection "A collection is an entity 
>>>>>>>> that has some members. The members are themselves entities").
>>>>>>>>
>>>>>>>> Renamed prov:membership to prov:qualifiedMembership to follow 
>>>>>>>> qualification pattern naming.
>>>>>>>> prov:Membership became subclass of prov:EntityInvolvement 
>>>>>>>> (though, it could become subclass of 
>>>>>>>> prov:KeyValuePairInvolvement, itself a subclass of 
>>>>>>>> prov:EntityInvolvement. But we'll try to simplify and reuse 
>>>>>>>> prov:entity)
>>>>>>>> prov:member renamed to prov:pair and became a subproperty of 
>>>>>>>> prov:involvee
>>>>>>>> Added property chain (qualifiedMembership o prov:pair) 
>>>>>>>> rdfs:subClassOf prov:hadMember
>>>>>>>> Added prov:removed domain prov:Removal range prov:KeyValuePair
>>>>>>>> Removed prov:CompleteDictionary from DM and PROV-O.
>>>>>>>>
>>>>>>>> Why?
>>>>>>>> Luc
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> What in particular would you like to discuss.
>>>>>>>> As I said, this reflects a response to many concerns that have 
>>>>>>>> been raised by many people in many forms.
>>>>>>>> In an effort to maintain focus and to make progress, I 
>>>>>>>> recommend that these points, the latest prov-dm, and the latest 
>>>>>>>> prov-o update serve as the basis for these discussions.
>>>>>>>>
>>>>>>>> -Tim
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> You'll notice the prov-o modeling of Dictionaries is not 
>>>>>>>> consistent with latest prov-dm.
>>>>>>>>
>>>>>>>> The prov-o team would like to ask the prov-dm editors to 
>>>>>>>> reconsider how collections and dictionaries are defined, so 
>>>>>>>> that they reflect the latest prov-o modeling of the PROV concepts.
>>>>>>>>
>>>>>>>> Regards,
>>>>>>>> Tim Lebo
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> cc tracker ISSUE-374 ISSUE-391
>>>>>>>>
>>>>>>>>
>>>>>>>>
>>>>>>>> --
>>>>>>>> Professor Luc Moreau
>>>>>>>> Electronics and Computer Science   tel:   +44 23 8059 4487
>>>>>>>> University of Southampton          fax:   +44 23 8059 2865
>>>>>>>> Southampton SO17 1BJ               email: 
>>>>>>>> l.moreau@ecs.soton.ac.uk <mailto:l.moreau@ecs.soton.ac.uk>
>>>>>>>> United Kingdom http://www.ecs.soton.ac.uk/~lavm 
>>>>>>>> <http://www.ecs.soton.ac.uk/%7Elavm>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> --
>>>>>>> Stian Soiland-Reyes, myGrid team
>>>>>>> School of Computer Science
>>>>>>> The University of Manchester
>>>>>>>
>>>>>>>
>>>>>>
>>>>>
>>>
>

-- 
Professor Luc Moreau
Electronics and Computer Science   tel:   +44 23 8059 4487
University of Southampton          fax:   +44 23 8059 2865
Southampton SO17 1BJ               email: l.moreau@ecs.soton.ac.uk
United Kingdom                     http://www.ecs.soton.ac.uk/~lavm