Re: Streaming and background information (was RSP next calls)

On 22 Nov 2013, at 08:00, Rinne Mikko <mikko.rinne@aalto.fi<mailto:mikko.rinne@aalto.fi>> wrote:

[snip]

4. Streaming and background information, hybrid objects

First-off, this morning I finally managed to answer "yes" to my old question in the wiki on whether we are going to have hybrid objects, i.e. objects that are both state objects and event objects at the same time. It is actually rather easy to come up with a scenario, where two or more stream producers will send state objects (temporally valid data), which will get single timestamps from an aggregator stream processing agent when merging the streams. And perhaps another set of timestamps assigned by a stream consumer upon reception. At this point every streaming object will contain both single timestamps and intervals and whether a streaming object is seen as a state object or event object no longer depends on the object itself, but rather on what a stream processing agent wants to do with it.

This is what I was looking for to break my earlier proposal, which was to have state objects and event objects as special cases of streaming objects. I'll try to merge the current state object and event object definitions somehow under the streaming object definition. As this is a bit bigger repair, I'll do it by deprecating the old "Elements in A Stream" section, copying into a new version of the section and editing that. We will then have the option of reverting to the old one, if the meeting thinks it was a bad idea or didn't come out right. I hope this change at least aligns with half of the comment from Emanuele requesting to only distinguish data as streaming and background.

On the other half of Emanuele's comment, "background information", I unfortunately somewhat disagree. To me "background information" stands for static datasets, which are typically retrieved all at once and can be processed with "normal" SPARQL semantics. I have no problem adding a definition for "background information" (we are contribution-driven! :-), but I wouldn't think of that as "an element in a stream". "Background" to me refers only to the data, not the method of delivery. The streaming of background data is possible, of course. Also, I would still keep the "static object" at least to:
a) indicate that we understand the difference
b) indicate that static objects can theoretically also be sent as a stream
c) be able to say what we don't do in the first phase.
As to problems with the word "static" I'm happy to discuss other proposals. My requirement would just be to keep it compact and understandable. To me "static" is fine with the interpretation "true until stated otherwise", which also differentiates it nicely from facts with temporal or instantaneous validity. "static or slowly changing" is too long. "semi-static" is ok, but in my opinion doesn't really add value in this case.

The issue here is that static (or stored) data is not really unchanging; particularly when you consider that streaming queries are long-lived.

Database management systems (I include triple stores in that definition) give the illusion that the data is static for the duration of a traditional query. This is how they are able to define the set of mappings that form the answer as the data is considered as a set at a point in time.

Continuous queries are evaluated periodically for a long period of time. During their execution the data that is stored may also change.

One proposal to overcome this is incorporated into the SNEEql query language [1]. There is an explicit window operator applied to the stored data that indicates the expected refresh rate of the stored data.

[snip]

Speak to you later

Alasdair

[1] C. Y. A. Brenninkmeijer, I. Galpin, A. A. A. Fernandes, and N. W. Paton, “A Semantics for a Query Language over Sensors, Streams and Relations,” in Proceedings of 25th British National Conference on Databases (BNCOD 25), 2008, vol. 5071, pp. 87–99. http://link.springer.com/chapter/10.1007%2F978-3-540-70504-8_9<http://link.springer.com/chapter/10.1007/978-3-540-70504-8_9>

Alasdair J G Gray
Lecturer in Computer Science, Heriot-Watt University, UK.
Email: A.J.G.Gray@hw.ac.uk<mailto:A.J.G.Gray@hw.ac.uk>
Web: http://www.macs.hw.ac.uk/~ajg33
ORCID: http://orcid.org/0000-0002-5711-4872
Telephone: +44 131 451 3429
Twitter: @gray_alasdair
Arrange a Meeting: http://doodle.com/agray

--

PLEASE NOTE: There may be a delay in me dealing with your email as I am participating in UCU industrial action by ‘working to contract’ in support of the union’s campaign for fair pay in higher education.
For more info go here www.ucu.org.uk/hepay13<http://www.ucu.org.uk/hepay13>






----- 
Sunday Times Scottish University of the Year 2011-2013
Top in the UK for student experience
Fourth university in the UK and top in Scotland (National Student Survey 2012)

We invite research leaders and ambitious early career researchers to 
join us in leading and driving research in key inter-disciplinary themes. 
Please see www.hw.ac.uk/researchleaders for further information and how
to apply.

Heriot-Watt University is a Scottish charity
registered under charity number SC000278.