Re: Negation over Streams

Hi

You are indeed correct with the problem you spot. I was thinking off the top of my head.

The issue is that you are trying to compute something over a pair of infinite streams; it is unlikely that we would have all the data we need in a single stream so this is a generally interesting problem.

The particular problem is that we can only know the scope of the second stream when we receive the full event on the first stream. Thus we need some way of scoping the window on the second stream based on the time interval covered in the first stream. This will probably require more complex window definition mechanisms and a history subquery over the second stream. I'm not entirely sure how to form that (and the bus ride home from work is not conducive to developing a solution).

Alasdair J G Gray
<http://www.alasdairjggray.co.uk><http://www.alasdairjggray.co.uk>http://www.alasdairjggray.co.uk


On 27 Feb 2015, at 15:30, Fariz Darari <fadirra@gmail.com<mailto:fadirra@gmail.com>> wrote:

Hi Alasdair,

following the RSP telco, if negation is performed over streams using
RDF triples, you suggested to do a join between two streams and then
run the negation over the windowed results of the join. This seems
good, but I have observed an issue that may occur:

Suppose Stream 1 is sending data of taxi rides' pickups and dropoffs
and Stream 2 is sending data of taxi rides' stops at traffic lights.
The streams are sent as RDF triples. The query is the same, taxi rides
that didn't meet any red traffic light. The join operation with
windowing over these two streams would be tricky as we don't know when
a taxi ride would finish. So, it can be the case that the window is
not long enough to capture the complete traffic light stops of taxi
rides (partial observation), thus may output incorrect query answers.
For this, I think we somehow need a way to keep track a single taxi
ride, which means the windowing is somehow
flexible. I am not sure if
this is already supported by current standards/proposals.

Let me know what you think. Thanks.

Regards,
Fariz


----- 
We invite research leaders and ambitious early career researchers to 
join us in leading and driving research in key inter-disciplinary themes. 
Please see www.hw.ac.uk/researchleaders for further information and how
to apply.

Heriot-Watt University is a Scottish charity
registered under charity number SC000278.