Re: FW: how to go from a list of lists to a list of first elements

Thanks for this Jos – it’s good to have extra examples. I collected the problematic collectAllIn cases here: https://docs.google.com/document/d/1RFp5vZ0Q1V6oZPXuXwjJDIILpgpz1hYWpjhZymdVcZE/edit?usp=sharing


FYI: in jen3, when the clause “(?S1 {?S :p ?S1} ?L2) log:collectAllIn ?Sc” is put in front of the premise, there is no issue: the builtin-internal "?S1" values are not exposed to rest of premise, so no problem if it is later unified with 9. If it is not, then jen3 will issue a warning that "?S1" is expected to be a variable but is a literal (as it was previously bound to 9 by the math:sum builtin). As I mentioned, jen3 is stricter in its expectations of the domain of builtins, in particular, collectAllIn. So, there is a problem with ordering in jen3 as well (which I expected since the current heuristic is quite simple).

So, in case of a certain ordering, and in a bit of a circuitous way, jen3 actually enforces the select variable to be local to the collectAllIn statement (as it issues a warning/error if the variable is already bound). It would be rather straightforward to have a separate check to enforce this in all cases.


William

From: Jos De Roo <josderoo@gmail.com>
Date: Thursday, January 20, 2022 at 12:15 PM
To: William Van Woensel <william.vanwoensel@gmail.com>
Cc: public-n3-dev@w3.org <public-n3-dev@w3.org>
Subject: Re: FW: how to go from a list of lists to a list of first elements
Hi William,

Well I suffered with all 15 cases and a particular (non-public) thorny one can be simplified to

:s1 :p 2, 3, 4.
:s2 :p 6, 7, 8, 9.
:s3 :p 8, 9, 10, 11.

{(?N1 {:s1 :p ?N1} ?L1) log:collectAllIn ?Sc. ?L1 math:sum ?S1. (?S1 {?S :p ?S1} ?L2) log:collectAllIn ?Sc} => {:test1 :result ?L2}.

In http://ppr.cs.dal.ca:3002/n3/editor/s/Ld8GINZx you can find the 6 permutations of the premis
and you can see that the results are contradictory.

What are the results you get with jen3?

Jos

-- https://josd.github.io



On Thu, Jan 20, 2022 at 3:28 PM William Van Woensel <william.vanwoensel@gmail.com<mailto:william.vanwoensel@gmail.com>> wrote:
Hi Jos

I’m sure there are cases where finding a solution is not possible. I’m just curious which cases made you decide that it was not a tractable problem


Thanks

William

From: Jos De Roo <josderoo@gmail.com<mailto:josderoo@gmail.com>>
Date: Wednesday, January 19, 2022 at 6:16 PM
To: William Van Woensel <william.vanwoensel@gmail.com<mailto:william.vanwoensel@gmail.com>>
Cc: Dörthe Arndt <doerthe.arndt@ugent.be<mailto:doerthe.arndt@ugent.be>>, public-n3-dev@w3.org<mailto:public-n3-dev@w3.org> <public-n3-dev@w3.org<mailto:public-n3-dev@w3.org>>
Subject: Re: FW: how to go from a list of lists to a list of first elements
Hi William,

Some weeks ago I investigated some 15 use cases that
use e:findall to do exactly what you ask. After spending
days and nights I was not able to find a suitable co-routining
solution. All I found was an experimental implementation
that was not compatible.

Jos

-- https://josd.github.io



On Wed, Jan 19, 2022 at 10:39 PM William Van Woensel <william.vanwoensel@gmail.com<mailto:william.vanwoensel@gmail.com>> wrote:
Hi Jos

Sorry for inaccurately formulating my question – I meant with regards to log:collectAllIn as this was the topic of my messages (but your e:optional example is useful nevertheless).

So, a bit more accurately: why wouldn’t it be possible to have a co-routining solution for log:collectAllIn – i.e., suspending execution until some condition is met (which for me involves clause re-ordering at runtime, really).

See my prior examples on this:
@prefix list: <http://www.w3.org/2000/10/swap/list#<http://www.w3.org/2000/10/swap/list>> .
@prefix log: <http://www.w3.org/2000/10/swap/log#<http://www.w3.org/2000/10/swap/log>> .
@prefix : <collectallin4.n3#> .
# won’t work (and will generate warnings for jen3, since the list:member builtin would have a mismatched domain, since “?members” isn’t bound):
{      ?l a :ConstantList .
       (?b { ?members list:member ?m . ?m :constant ?b } ?booleans ) log:collectAllIn _:x .
       (?m { ?l list:member ?m . ?m :constant true } ?members) log:collectAllIn _:x .
} => { :test2 a ?members , ?booleans } .
# will work
{      ?l a :ConstantList .
       (?m { ?l list:member ?m . ?m :constant true } ?members) log:collectAllIn _:x .
       (?b { ?members list:member ?m . ?m :constant ?b } ?booleans ) log:collectAllIn _:x .
} => { :test2 a ?members , ?booleans } .
Although I can see solutions for this particular case that would require some more introspection ..



Thanks

William

From: Jos De Roo <josderoo@gmail.com<mailto:josderoo@gmail.com>>
Date: Wednesday, January 19, 2022 at 5:21 PM
To: William Van Woensel <william.vanwoensel@gmail.com<mailto:william.vanwoensel@gmail.com>>
Cc: Dörthe Arndt <doerthe.arndt@ugent.be<mailto:doerthe.arndt@ugent.be>>, public-n3-dev@w3.org<mailto:public-n3-dev@w3.org> <public-n3-dev@w3.org<mailto:public-n3-dev@w3.org>>
Subject: Re: FW: how to go from a list of lists to a list of first elements
Hi William,

From the 265 built-ins that eye supports some 50 built-ins
are without co-routining simply because we were not able
to define a condition to suspend their execution till that
condition is met. Examples are log:equalTo, log:notEqualTo,
log:implies, e:becomes, e:findall, e:derive, e:optional, etc..

So using those built-ins can lead to rules that are premis
order dependent, like you can see in the simple example
http://ppr.cs.dal.ca:3002/n3/editor/s/sue24Zdt


Jos

-- https://josd.github.io



On Wed, Jan 19, 2022 at 4:24 PM William Van Woensel <william.vanwoensel@gmail.com<mailto:william.vanwoensel@gmail.com>> wrote:
Hi Jos,

There are examples where the order of built-ins really matters for EYE

                I’d be interested in seeing those examples, and why Eye struggles with them.


William

On Tue, Jan 11, 2022 at 7:35 PM William Van Woensel <william.vanwoensel@gmail.com<mailto:william.vanwoensel@gmail.com>> wrote:
For instance, an example where the simple clause re-ordering currently doesn’t work (as both builtin statements would be biased to the end of the clause, their original ordering would be kept):



@prefix list: <http://www.w3.org/2000/10/swap/list#<http://www.w3.org/2000/10/swap/list>> .

@prefix log: <http://www.w3.org/2000/10/swap/log#<http://www.w3.org/2000/10/swap/log>> .

@prefix : <collectallin4.n3#> .

# won’t work (and will generate warnings for jen3, since the list:member builtin would have a mismatched domain, since “?members” isn’t bound):

{      ?l a :ConstantList .

       (?b { ?members list:member ?m . ?m :constant ?b } ?booleans ) log:collectAllIn _:x .

       (?m { ?l list:member ?m . ?m :constant true } ?members) log:collectAllIn _:x .



} => { :test2 a ?members , ?booleans } .



# will work

{      ?l a :ConstantList .

       (?m { ?l list:member ?m . ?m :constant true } ?members) log:collectAllIn _:x .

       (?b { ?members list:member ?m . ?m :constant ?b } ?booleans ) log:collectAllIn _:x .

} => { :test2 a ?members , ?booleans } .
(the internal “?m” variable in the where clause isn’t exposed to the outside, so that one is not a problem..)

Although I can see solutions for this particular case that would require some more introspection ..


William

From: William Van Woensel <william.vanwoensel@gmail.com<mailto:william.vanwoensel@gmail.com>>
Date: Tuesday, January 11, 2022 at 2:15 PM
To: Doerthe Arndt <doerthe.arndt@tu-dresden.de<mailto:doerthe.arndt@tu-dresden.de>>
Cc: public-n3-dev@w3.org<mailto:public-n3-dev@w3.org> <public-n3-dev@w3.org<mailto:public-n3-dev@w3.org>>
Subject: Re: how to go from a list of lists to a list of first elements
Hi Doerthe,

The built-ins log:collectAllIn and e:findall are order dependent, i.e. it depends on the order of the triples in the premise of a rule whether a variable is considered as local for the built-in or whether it is bound by its context. A simple example would be the following: http://ppr.cs.dal.ca:3002/n3/editor/s/6XTgh8ru<http://ppr.cs.dal.ca:3002/n3/editor/s/FufmKC1w>

                Yes, I found this problem quite early on and tried solving it using a clause re-ordering

We came up with the idea that locally scoped variables (like the ?task in the example) should be explicitly declared. Now, Jos solved that in his implementation by using the ?select-part (remember the structure ?SCOPE e:findall (?SELECT ?WHERE ?ANSWER).) can be used for that declaration.

                Yes I thought I was missing a step or two in that discussion :-) Still, this is very unclear to me. What what would happen in this case:

                { ?x a :task.
 ((?task ?u)  {?x :subTask ?task. ?x :marked ?u} ?ys) log:collectAllIn ?Scope.
 ?u log:equalTo true.
}
=> { ?x :list1 ?ys }.

Would these two “?u”s simply not co-refer, would some variable re-naming take place, would an error be thrown, ..?

How about this:

{ ?x a :task.
 (?task  {?x :subTask ?task. ?x :marked ?u} ?ys) log:collectAllIn ?Scope.
}
=> { ?x :list1 ?ys }.

Would an error be thrown since “?u” is not grounded? Would it be fine if a blank node is used?

In general I dislike how these solutions are putting the onus on the N3 author to solve the problem for us. Solving this for all cases with an internal rule re-ordering may be impossible (as I myself tried to point out before) but I vastly prefer it at this point.

Here, it depends on the outcome of our current discussion about the semantics (do we have two different kinds variables or just one) whether that is the case or not.

                As I mentioned before, I found this “mechanism” (i.e., creating rules by composing the antecedent from multiple graphs) to be very useful to modularize code – see here<http://ppr.cs.dal.ca:3002/n3/editor/s/RW8UEsk1> for a simplified example (it doesn’t work in eye, currently).


Regards,

William

From: Doerthe Arndt <doerthe.arndt@tu-dresden.de<mailto:doerthe.arndt@tu-dresden.de>>
Date: Tuesday, January 11, 2022 at 10:46 AM
To: Doerthe Arndt <doerthe_arndt@yahoo.de<mailto:doerthe_arndt@yahoo.de>>
Cc: public-n3-dev@w3.org<mailto:public-n3-dev@w3.org> <public-n3-dev@w3.org<mailto:public-n3-dev@w3.org>>
Subject: Re: how to go from a list of lists to a list of first elements
Hi William,

I will answer your two questions.

First about yesterday’s discussion (just a summary since Jos already answered):

Original problem:
The built-ins log:collectAllIn and e:findall are order dependent, i.e. it depends on the order of the triples in the premise of a rule whether a variable is considered as local for the built-in or whether it is bound by its context. A simple example would be the following: http://ppr.cs.dal.ca:3002/n3/editor/s/6XTgh8ru<http://ppr.cs.dal.ca:3002/n3/editor/s/FufmKC1w>
In the old implementation of EYE, you would have gotten


:cmp0 :list1 (:task99). :cmp1 :list1 (:task98). :cmp0 :list2 (:task99). :cmp1 :list2 ().

The important part is the second triple which is different in that case.

Solution:
We came up with the idea that locally scoped variables (like the ?task in the example) should be explicitly declared. Now, Jos solved that in his implementation by using the ?select-part (remember the structure ?SCOPE e:findall (?SELECT ?WHERE ?ANSWER).) can be used for that declaration.

Concern:
While I liked the idea in general, I disliked that it means that I also have to put variables in the select-pattern which I don’t use for further reasoning. That would make it difficult to access the result I am actually interested in (Jos gave an example below which was based on your example).

Possible solution:
I thought it would help to have good built-ins to deal with the lists resulting from such findall-queries with nested lists and therefore Jos gave a solution.


Now, back to your original question (which is related). I guess in the result you get from here http://ppr.cs.dal.ca:3002/n3/editor/s/MyvdkJpa which is (just to be sure):

{?U_0 a :Mouse. ?U_0 log:equalTo :Jerry. (?U_1 {?U_1 a :Dog. ?U_1 :hates ?U_2} ?U_3) log:collectAllIn _:e_x_1} => {:test2 :success true}.

You would like to get that ?U_0 and ?U_2 are the same variables?

Here, it depends on the outcome of our current discussion about the semantics (do we have two different kinds variables or just one) whether that is the case or not.

Kind regards,
Dörthe






----- Weitergeleitete Nachricht -----
Von: Jos De Roo <josderoo@gmail.com<mailto:josderoo@gmail.com>>
An: William Van Woensel <william.vanwoensel@gmail.com<mailto:william.vanwoensel@gmail.com>>
CC: public-n3-dev@w3.org<mailto:public-n3-dev@w3.org> <public-n3-dev@w3.org<mailto:public-n3-dev@w3.org>>
Gesendet: Dienstag, 11. Januar 2022, 14:29:08 MEZ
Betreff: Re: how to go from a list of lists to a list of first elements

Yes indeed William, http://ppr.cs.dal.ca:3002/n3/editor/s/MyvdkJpa does not work
but http://ppr.cs.dal.ca:3002/n3/editor/s/qr0in3if works fine.

The reason is that the experimental implementation of log:collectAllIn expects
that all the universals in the where-clause that are not in the select-pattern
must be ground. The built-in is eventually suspended to achieve this grounding.

This is also the reason that Doerthe was not happy with the extended select-pattern
but the answer-list can be simplified again using list:map as shown in the previous email.

Jos

-- https://josd.github.io<https://josd.github.io/>


On Tue, Jan 11, 2022 at 1:58 PM William Van Woensel <william.vanwoensel@gmail.com<mailto:william.vanwoensel@gmail.com>> wrote:

Hi Jos, Doerthe,

My confusion about this example yesterday stemmed from the fact that it is unclear why “?u” is also being collected, if it is not being used later on. But now I assume it is simply part of the example (and not related to scoping within the collectAllIn builtin, which is what I believe we were talking about :-)?



On that note: as I mentioned yesterday, one could identify at compile time all local variables within the where clause of collectAllIn, but only barring some exceptional circumstances. Such as:

http://ppr.cs.dal.ca:3002/n3/editor/s/tfsdyPr0




(this “pulling in” of universal variables into rules, meaning they become part of the reasoning machinery, was actually a big problem in jen3..)



When compiling this example I found that the following doesn’t seem to work in Eye:

http://ppr.cs.dal.ca:3002/n3/editor/s/zc2tHpTw






Thoughts?



William



From: Jos De Roo <josderoo@gmail.com<mailto:josderoo@gmail.com>>
Date: Monday, January 10, 2022 at 3:03 PM
To: public-n3-dev@w3.org<mailto:public-n3-dev@w3.org> <public-n3-dev@w3.org<mailto:public-n3-dev@w3.org>>
Subject: how to go from a list of lists to a list of first elements

This is about the question of Doerthe

[[
Question:
http://ppr.cs.dal.ca:3002/n3/editor/s/Mut5EohI

Would we use built-ins here?

]]



So how to go from a list of lists to a list of first elements?



We could use list:map like is done in http://ppr.cs.dal.ca:3002/n3/editor/s/xrtMG2Xc




Jos

-- https://josd.github.io<https://josd.github.io/>