- From: Manfred Hauswirth <manfred.hauswirth@deri.org>
- Date: Fri, 22 Nov 2013 10:56:34 +0000
- To: public-rsp@w3.org
Hi Rinne, all,
> a) We have no definition of "time-series". The problem with the one
> given in email ("a non decreasing timestamp") is that it effectively
> hides all problematics of a distributed system, where streaming objects
> may both have timestamps assigned by non-synchronized clocks and the
> objects may arrive to the stream processor out-of-order. Restricting to
> non-decreasing timestamps would effectively limit us to cases, where
> timestamps are assigned by a singular place per stream (a stream
> processing agent). Ability to properly support distributed,
> heterogeneous environments is a big motivator for using RDF in the first
> place. Therefore I fear that by restricting ourselves to non-decreasing
> timestamps we lose more than may first appear.
+1
> I'm still uncertain about whether to include implicit timestamps to the
> scope. On one hand it seems overkill to require every low-level
> streaming object to have an explicit timestamp, especially if the next
> stream processing agent on the path is going to assign it a new one
> anyway, but on the other hand it is hard to imagine a device capable of
This may be the case but cannot be required - and may not be necessary.
This is a parameter that any deployment may decide on implicitely.
> producing RDF without any kind of clock circuit. Perhaps it is not so
> much the capability of the devices, but rather the extra trouble of
> trying to keep the clocks of e.g. 5,000 sensors in sync, if the
> transmission network to the stream aggregator has very low delay
> variation. As a result I still believe that implicit timestamps should
> be in the scope, but it would be fairly easy convince me otherwise.
Distributed time sync is one of the hardest problem in CS. We can and
will not solve it in this XG. What we need is a way of modelling various
models to capture time and the let the processor (under guidance of the
user) decide how to handle it (and this will differ from system to
system). E.g., one application may require re-calculations while in
another application the late arriving value may be treated as irrelevant
and be thrown away. We should not try to support everything
automatically - that usually leads to telephone book standard that
nobody can use. Pragmatics over completeness, IMHO.
> b) "Infinite" rules out recorded streams. Since recorded streams are the
> most verifiable part of this work, it seems strange to proceed without
> them. Also, I'm not sure how the inclusion of recorded streams would
> slow down the work, because a segment of a live stream can always be
> recorded and played back. The biggest difference is that a recorded
> stream requires explicit knowledge of timestamps, but if our live stream
> work acknowledges the presence of explicit timestamps, the "feature" is
> already included by default.
I do not understand why this is an issue. If something is recorded and
replayed or "infinite" is a mere definition and of mere theoretical
interest that should not change a model to be developed.
> *2. Ordered / not ordered*
>
> We had some discussion with Alasdair on the wiki, but there's not a
> proposal on the table (afaik) for what to write into the
> characterization of a recorded stream. For me the issue of ordering
> itself is relative: Any stream processing agent (including stream
> producers) can insert a counter, which can be used to restore order
> observed at that point in time. Whether that is the "correct" order, is
> a much more philosophical question. In theory the "correct" order of
> observations is the one in which the observations were made, but due to
> system inaccuracies we may not have the data to restore that order. Is
> our stream ordered if the timestamps are properly increasing, but the
> sound of lightning is observed before the flash?
Again - that depends on the application semantics: If you calculate an
average of something which does not change quickly, order is of low
relevance. If your data item order is important to describe
dependencies, the the game changes. But, why are we discussing things
here that distributed systems guys have discussed for 30 years? We
should not enshrine a specific model but rather allow the user to model
everything as needed which guidance what that means for the results that
the processor produces. There is no wrong or right - it depends on what
level of consistency an application needs or can tolerate.
> Without other proposals on the table my suggestion would be to remove
> "ordered / not ordered" from the definition of recorded streams (it
> doesn't differentiate recorded from live - live can also arrive
> out-of-order), and instead write something about the capability of the
> stream to support order restoration once we get to requirements.
I do not see any conceptual difference between a live of recorded
stream. Why differentiate? Am I missing something?
> *3. Streams and stream elements lacking reference to RDF*
>
> Emanuele was commenting on this. I can, of course, do the following
> replacements:
>
> Streams => RDF Streams
>
> (Live) Data Stream: An unbounded sequence of time-varying data elements.
> => (Live) RDF Data Stream: An unbounded sequence of time-varying data
> elements encoded in RDF.
>
> Recorded Data stream: A stream saved e.g. to a computer file.
> => Recorded RDF Data stream: An RDF stream saved e.g. to a computer file.
>
> Elements in A Stream => Elements in An RDF Stream
>
> ...etc. etc. But is there something more substantial that we need to
> say? Does it add value to insert RDF everywhere, or is it enough that
> the group scope is RDF streams?
RDF is just a format IMHO that allows you to easily integrate the data
and results with other data. Nothing more. All streams are the same in
whatever notation they come. What RDF gives you is a general framework
and a nice general data model, e.g., a graph which changes its shape and
constituting data over time and RDF as a nice formalism to capture this.
> *4. Streaming and background information, hybrid objects*
>
> First-off, this morning I finally managed to answer "yes" to my old
> question in the wiki on whether we are going to have hybrid objects,
> i.e. objects that are both state objects and event objects at the same
> time. It is actually rather easy to come up with a scenario, where two
> or more stream producers will send state objects (temporally valid
> data), which will get single timestamps from an aggregator stream
> processing agent when merging the streams. And perhaps another set of
> timestamps assigned by a stream consumer upon reception. At this point
> every streaming object will contain both single timestamps and intervals
> and whether a streaming object is seen as a state object or event object
> no longer depends on the object itself, but rather on what a stream
> processing agent wants to do with it.
=> This is what I mean by application semantics. The model should allow
you to do this.
> This is what I was looking for to break my earlier proposal, which was
> to have state objects and event objects as special cases of streaming
> objects. I'll try to merge the current state object and event object
> definitions somehow under the streaming object definition. As this is a
> bit bigger repair, I'll do it by deprecating the old "Elements in A
> Stream" section, copying into a new version of the section and editing
> that. We will then have the option of reverting to the old one, if the
> meeting thinks it was a bad idea or didn't come out right. I hope this
> change at least aligns with half of the comment from Emanuele requesting
> to only distinguish data as streaming and background.
>
> On the other half of Emanuele's comment, "background information", I
> unfortunately somewhat disagree. To me "background information" stands
> for static datasets, which are typically retrieved all at once and can
> be processed with "normal" SPARQL semantics. I have no problem adding a
> definition for "background information" (we are contribution-driven!
> :-), but I wouldn't think of that as "an element in a stream".
> "Background" to me refers only to the data, not the method of delivery.
> The streaming of background data is possible, of course. Also, I would
> still keep the "static object" at least to:
> a) indicate that we understand the difference
> b) indicate that static objects can theoretically also be sent as a stream
> c) be able to say what we don't do in the first phase.
> As to problems with the word "static" I'm happy to discuss other
> proposals. My requirement would just be to keep it compact and
> understandable. To me "static" is fine with the interpretation "true
> until stated otherwise", which also differentiates it nicely from facts
> with temporal or instantaneous validity. "static or slowly changing" is
> too long. "semi-static" is ok, but in my opinion doesn't really add
> value in this case.
Why is this important how we call it? Whether something is "background"
is in the eye of the beholder, i.e., the application designer, IMHO. All
we need is streams + static data IMHO. If you send static data in a
streaming fashion (e.g. for efficiency reasons), it becomes stream data.
We are splitting hairs here. Let's be pragmatic.
> *5. Time as annotation vs. time as data*
>
> The intention with defining the streaming objects was to encapsulate
> both. I'll try to include the annotation aspect of time when updating
> the streaming object definition. At the same time I'm trying to avoid
> building assumptions of the solution into the definitions, because these
> definitions are only there to help us write requirements and should not
> prematurely fix solutions.
The same issue applies to location. You should be able to model it as
meta-data (static sensor) or as data (moving sensor).
Best,
Manfred
>
> --------------------------
>
> Those were the things I had in mind, next I'll try to work a bit on the
> Wiki. One observation is that we haven't had many proposals for new
> definitions since the initial set. It's nice if we can manage with
> these, but more likely we will just have to revisit this document once
> we start work on the requirements and see what we really need.
>
> BR,
>
> Mikko
>
> On 14. Nov 2013, at 4:41 PM, Emanuele Della Valle wrote:
>
>> Hello Jean-Paul, Eva, Avi and all,
>>
>> what Eva says and Jean-Paul confirm is also what I think, sorry to
>> create confusions.
>>
>> My answer to Avi was putting emphasis on the fact that by
>> "time-series" I do not mean a *sequence of numbers ordered by
>> recency*, but a *sequence of RDF triples ordered by recency*.
>>
>> Concerning how to describe time from the application perspective, my
>> position is the following one:
>> - 0 timestamps (i.e., relying on the temporal distance between the
>> received triples ) makes compatibility with RDF straight forward, but
>> it may hide problems (e.g., the temporal distance between two triples
>> may be influenced by network delays)
>> - 1 (point in time semantics for application time) allows for handling
>> out of orders and for basic temporal operators (e.g., follows,
>> precedes, contemporaryWith)
>> - 2 (interval based semantics for events) allows for expressive
>> temporal operators, but, at least in many scenarios I target, it is an
>> overkill
>>
>> Most of the commercial DSMS/CEP take either 0 or 1. The only
>> commercial CEP that I know supporting 2 is Microsoft StreamInsight.
>>
>> Time from the system perspective is a different issue. Whether system
>> time should be externalised is something I still wonder.
>>
>> Cheers,
>>
>> Emanuele
>>
>>
>> On Nov 14, 2013, at 1:58 AM, Jean-Paul <jp.calbimonte@upm.es
>> <mailto:jp.calbimonte@upm.es>>
>> wrote:
>>
>>> Hello all,
>>>
>>> Yes, I think Oscar's diagram (check it here:
>>> http://www.w3.org/community/rsp/wiki/Meeting_22.10.2013) more or less
>>> reflects part of the discussion we had about the scope.
>>>
>>> We seem to agree that ordered streams of elements (infinite or
>>> 'recorded' streams as well) are in scope (green ticks in the
>>> diagram). In these cases the order might be of different natures but
>>> we agreed to focus on time-based order. I don't think we agreed yet
>>> on focusing only in point in time timestamps or intervals. For the
>>> moment it is just time-based order, I believe.
>>>
>>> Then there are datasets which may not be streaming in nature but
>>> might be needed to processed in a streaming fashion (e.g. a very
>>> large dataset). I understood we are not ruling this case out, but
>>> might not focus on it in a first stage.
>>>
>>> Thanks to Emanuele for the input about the scope. As Eva pointed out,
>>> there are some discrepancies that we can fix in the wiki. I am also a
>>> bit unconfortable with calling the streams in our scope as
>>> 'time-series', I think this term has other connotations in related areas.
>>>
>>> well, this is just a personal comment as well, but I'm happy to
>>> continue this discussion. We can also continue modifying the wiki
>>> until we have the Telco, and afterwards.
>>>
>>> best
>>> jp
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>>
>>> 2013/11/13 Eva Blomqvist <evabl444@gmail.com <mailto:evabl444@gmail.com>>
>>>
>>> Hi!
>>> I think that some of those who were in the meeting also might
>>> have slightly differing interpretations of what was said... I
>>> agree to that there were two alternative interpretations of "data
>>> stream" discussed, but as far as I understood those differed in
>>> the sense that 1) was an *infinite* stream, where the elements of
>>> the stream could somehow be *associated with time* (whether a
>>> timestamped triple, a timestamped graph, or just a stream where
>>> time is implicit from the arrival times of elements etc), and 2)
>>> was a *finite* stream of elements where *time is not necessarily
>>> an aspect*, e.g. triples from a data store being processed in a
>>> streaming fashion.
>>>
>>> I would be reluctant to at this stage limit ourselves to a
>>> specific model, e.g. RDF statements with a single timestamp each.
>>> Just my 2c..
>>> /Eva
>>>
>>>
>>> On 12/11/2013 17:33 , Emanuele Della Valle wrote:
>>>> Hi Abram,
>>>>
>>>> I mean a list of tuples <s,t> where s is an RDF statement and t
>>>> is a non decreasing timestamp.
>>>>
>>>> Cheers,
>>>>
>>>> Emanuele
>>>>
>>>> --
>>>> prof. Emanuele Della Valle
>>>> DEIB - Politecnico di Milano
>>>> m. +393389375810 <tel:%2B393389375810>
>>>> w. http://emanueledellavalle.org <http://emanueledellavalle.org/>
>>>>
>>>> On Nov 12, 2013, at 12:27 PM, Abraham Bernstein
>>>> <bernstein@ifi.uzh.ch <mailto:bernstein@ifi.uzh.ch>>
>>>> wrote:
>>>>
>>>>> Emanuele, all
>>>>>
>>>>> I am slightly confused.... so just to clarify When you talk
>>>>> about time-series: do you mean a series of numbers (expressed
>>>>> in triples) or a time-ordered series of triples?
>>>>>
>>>>> Cheers
>>>>>
>>>>> Avi
>>>>>
>>>>>
>>>>> On 12.11.2013, at 03:05, Jean-Paul <jp.calbimonte@upm.es
>>>>> <mailto:jp.calbimonte@upm.es>> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> Thanks for your input. 4th Telco will be on nov 22 15:00 CET.
>>>>>> We will be discussing about the Streams concepts and
>>>>>> definitions that we have started drafting in the wiki.
>>>>>> Please feel free to provide your input there already:
>>>>>>
>>>>>> http://www.w3.org/community/rsp/wiki/Concepts_and_Definitions
>>>>>>
>>>>>> ...specialy if there is a key concept missing that you
>>>>>> consider we should include.
>>>>>>
>>>>>> Cheers,
>>>>>> jp
>>>>>>
>>>>>>
>>>>>> PS
>>>>>> Please, if Danh or Manfred can help us again with Webex, we
>>>>>> will be very thankful.
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> 2013/11/6 Jean-Paul <jp.calbimonte@upm.es
>>>>>> <mailto:jp.calbimonte@upm.es>>
>>>>>>
>>>>>> Yes, I see. That will make everyone's life easier.
>>>>>> We'll dicuss it.
>>>>>>
>>>>>> thanks
>>>>>>
>>>>>> jp
>>>>>>
>>>>>>
>>>>>> 2013/11/6 Axel Polleres <axel@polleres.net
>>>>>> <mailto:axel@polleres.net>>
>>>>>>
>>>>>> Thanks, BTW, may I suggest that instead of a single
>>>>>> doodle per Telco, to doodle for one fixed timeslot per
>>>>>> week, e.g. "Tue 15:00" or alike, as usual in other
>>>>>> WGs? I think this should make planning easier. Maybe
>>>>>> we can discuss this in the Telco.
>>>>>>
>>>>>> thanks & best regards,
>>>>>> Axel
>>>>>>
>>>>>> --
>>>>>> Prof. Dr. Axel Polleres
>>>>>> Institute for Information Business, WU Vienna
>>>>>> url: http://www.polleres.net/ twitter: @AxelPolleres
>>>>>>
>>>>>> On Nov 2, 2013, at 11:40 PM, Jean-Paul
>>>>>> <jp.calbimonte@upm.es <mailto:jp.calbimonte@upm.es>>
>>>>>> wrote:
>>>>>>
>>>>>> > Hello All,
>>>>>> >
>>>>>> > Thanks to all who could attend the meeting at ISWC,
>>>>>> and specially to those who made it through WebEx
>>>>>> (although couldn't interact too much, unfortunately)
>>>>>> >
>>>>>> > The meeting went quite well, and we received input
>>>>>> from people of other sub-communities and with
>>>>>> different background. Others showed interest, at least
>>>>>> as 'observers' of what we are trying to do.
>>>>>> >
>>>>>> > One result of the meting is the intention of
>>>>>> clarifying the scope of our work. A first step to do
>>>>>> this is to have written some of the key concepts and
>>>>>> definitions that we should agree on. Mikko has already
>>>>>> provided a first version as he already commented, and
>>>>>> the purpose of the next telecon will be to discuss them:
>>>>>> >
>>>>>> >
>>>>>> http://www.w3.org/community/rsp/wiki/Concepts_and_Definitions
>>>>>> >
>>>>>> > Until then, I invite you all to contribute to that (
>>>>>> I see some have already started, great!) so that we
>>>>>> can have material for discussion.
>>>>>> >
>>>>>> > Please, also indicate your preferences for the next
>>>>>> calls:
>>>>>> >
>>>>>> > http://doodle.com/a8ggni2v4su7c88b
>>>>>> >
>>>>>> > http://doodle.com/6i97qvmaqiwnwvsa
>>>>>> >
>>>>>> > http://doodle.com/hixgfbv9drxbu4in
>>>>>> >
>>>>>> >
>>>>>> > Thanks to all,
>>>>>> >
>>>>>> > jp
>>>>>> >
>>>>>> > --
>>>>>> > Jean-Paul Calbimonte
>>>>>> > Ontology Engineering Group
>>>>>> > Universidad Politécnica de Madrid
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jean-Paul Calbimonte
>>>>>> Ontology Engineering Group
>>>>>> Universidad Politécnica de Madrid
>>>>>>
>>>>>>
>>>>>>
>>>>>>
>>>>>> --
>>>>>> Jean-Paul Calbimonte
>>>>>> Ontology Engineering Group
>>>>>> Universidad Politécnica de Madrid
>>>>>
>>>>> -----------------------------------------------------------------
>>>>> | Professor Abraham Bernstein, PhD
>>>>> | University of Zürich, Department of Informatics
>>>>> | web: http://www.ifi.uzh.ch/ddis/bernstein.html
>>>>>
>>>>
>>>
>>>
>>>
>>>
>>> --
>>> Jean-Paul Calbimonte
>>> Ontology Engineering Group
>>> Universidad Politécnica de Madrid
>>
>
--
Prof. Manfred Hauswirth
Digital Enterprise Research Institute (DERI)
National University of Ireland, Galway (NUIG)
http://www.manfredhauswirth.org/
Received on Friday, 22 November 2013 10:57:03 UTC