Re: AW: data streams

Hi all,

I agree that pub-sub model for handling this kind of streams of data 
makes a lot of sense.

For folks not familiar with the IETF work, here's pointers to relevant 
drafts.

Observing Resources in CoAP:
https://tools.ietf.org/html/draft-ietf-core-observe-16

Publish-Subscribe Broker for CoAP:
https://tools.ietf.org/html/draft-koster-core-coap-pubsub-01

The observe draft is in final stages to become an RFC while the pub-sub 
broker draft is still in early stages of specification.

I'm one of the co-authors of the pub-sub broker draft and we are 
currently updating it for the next IETF meeting. Comments on that are 
very welcome.


Cheers,
Ari


On 03/06/15 06:12, Hund, Johannes wrote:
> Hi Magnus,
>
> in the discussions @IETF there are basically two models being discussed,
> even there the discussion is quite related  with the used protocol
> (CoAP), the concepts are remarkable generic.
>
> There one Architecture model is having a central pub-sub broker (can
> also be “central”  for a small piece of the application such as a room
> or a home), while the other is more generalized to have a way to
> orchestrate data bindings, i.e. to tell one resource to convey data
> changes to or from another resource. That means I could tell one
> resource to observe another resource (and optionally tell it to change
> its state accordingly) or I can tell one resource to push any data
> changes to another resource. The pub-sub broker would be a special case
> of that.
>
>  From the current discussions of the WoT framework e.g. between Vlad and
> me, I would see this also covered by the current approach, as one
> “thing” can have properties and you should be able to subscribe to a
> property to get informed on data change.  How this subscription is done
> is dependent on the protocol that the framework resp. model is mapped to.
>
> Same goes with events, the main difference between a property change and
> an event is that the event might have a greater semantic meaning e.g. a
> flatline of the ECG or e.g. might be referring to several properties,
> such as heart rate above a threshold value.
>
> I really think that the model we currently are discussing is capable of
> providing these use cases Dave mentioned, but we should define
> higher-level patterns (e.g. sensor value  + threshold as properties and
> an event source for threshold traversal or denying subscriptions if
> there is already one active subscription) based on the framework that
> could e.g. have own APIs or be described as such in the TDL. That is for
> example how the CoRE interfaces are done in the IETF, they are basically
> interaction patterns of a RESTful API.
>
> I was not quite sure in the discussion below: is there the suggestion to
> forsee a way to manage out-of-band streams or is it referring to inband?
>
> Also it might be useful, I am not sure if we should address management
> of out-of-band streams, as it opens quite a can of worms. If the
> consensus is we do need it, then I would see it sufficient to also
> provide a sort of pattern as mentioned above.
>
> Any thoughts?
>
> Best Regards,
>
> Johannes
>
> *Von:*Magnus Olsson [mailto:magnus.olsson@ericsson.com]
> *Gesendet:* Mittwoch, 3. Juni 2015 10:27
> *An:* Edoardo Pignotti; public-wot-ig@w3.org
> *Betreff:* Re: data streams
>
> Edoardo,
>
> Great if it works I have no practical experience myself using the
> pub-sub for IoT data but understand that there is standards work (in
> IETF) that could be possible to reference. What is the Oxford flood
> network, sounds like an Iot mesh?
>
> Br magnus
>
> http://www.ericsson.com/current_campaign
> <http://www.ericsson.com/current_campaign>
>
> *MAGNUS OLSSON *
>
>
> Ericsson AB
> Group Function Technology
> Mobilvägen 1
> 22362 Lund, Sweden
> SMS/MMS +46 706 756814
> magnus.olsson@ericsson.com <mailto:magnus.olsson@ericsson.com>
> www.ericsson.com <http://www.ericsson.com>
>
>
>
> Ericsson
>
>
>
> This Communication is Confidential. We only send and receive email on
> the basis of the terms set out at www.ericsson.com/email_disclaimer
> <http://www.ericsson.com/email_disclaimer>
>
> On 2015-06-03 10:19, Edoardo Pignotti wrote:
>
>     Hello,
>
>     I also agree that streams would be a very useful thing to support in
>     the WoT framework.
>
>     I am in favour of a pub-sub broker model. In the oxford flood
>     network we thought very hard how to manage data streams and we went
>     for a pub-sub model.
>
>     I would be more than happy to discuss this further in the context of
>     our case-study.
>
>     All the best,
>
>     Edoardo
>
>     *From: *Magnus Olsson <magnus.olsson@ericsson.com
>     <mailto:magnus.olsson@ericsson.com>>
>     *Organization: *Ericsson AB
>     *Date: *Wednesday, 3 June 2015 09:04
>     *To: *"public-wot-ig@w3.org <mailto:public-wot-ig@w3.org>"
>     <public-wot-ig@w3.org <mailto:public-wot-ig@w3.org>>
>     *Subject: *Re: data streams
>     *Resent-From: *<public-wot-ig@w3.org <mailto:public-wot-ig@w3.org>>
>     *Resent-Date: *Wednesday, 3 June 2015 09:05
>
>     Dave,    (and woters)
>
>     Streams sounds like a very useful thing to support :-)
>
>     - For the multiple sinks, why not use a pub sub broker as the
>     identified source of the stream that way any number of sinks can
>     attach to that pub-source. Seems to me that setting a proxy output
>     value to point to a single output is a limitation that might result
>     in someone creating an upstream proxy to split to multiple sinks.
>
>     Any thing should be able to redirect any aspect of their source
>     (defined by its meta description) to a nearby pub-sub proxy service.
>     Still that could be an "on device" (localhost) broker, a home
>     broker, a service provider broker or a public broker etc.
>
>     It seems reasonable to me to have use case such as one part of the
>     stream for the snapshot, another for the live view a third for data
>     analysis, a fourth for a remote tele medicine, a fifth for a
>     emergency monitoring function etc.
>
>     - Even discrete value type of sensors could be handled as a stream
>     if only given long enough time to observe
>
>     - (Not convinced myself) but perhaps any data points should be
>     allowed to be retrieved as a stream with sometime only a single
>     value being sent before the connection is being terminated.
>
>     - If there is a possibility to "cascade" (or chaining) the streamed
>     data points then it might open up for a "network of" data stream
>     functions that could apply smartness to the stream such as adding
>     timestamps (simple sensors do not have timestamps, but sometimes
>     have sample rates), compression, average, de-noising, composite streams.
>
>     Thanks!
>
>
>     Br magnus
>
>     http://www.ericsson.com/current_campaign
>     <http://www.ericsson.com/current_campaign>
>
>     *MAGNUS OLSSON *
>
>
>     Ericsson AB
>     Group Function Technology
>     Mobilvägen 1
>     22362 Lund, Sweden
>     SMS/MMS +46 706 756814
>     magnus.olsson@ericsson.com <mailto:magnus.olsson@ericsson.com>
>     www.ericsson.com <http://www.ericsson.com>
>
>
>
>     Ericsson
>
>
>
>     This Communication is Confidential. We only send and receive email
>     on the basis of the terms set out at
>     www.ericsson.com/email_disclaimer
>     <http://www.ericsson.com/email_disclaimer>
>
>     On 2015-06-03 09:16, Dave Raggett wrote:
>
>         I am interested in enabling the Web of Things Framework to
>         handle data streams.
>
>         Some sensors produces streams of data as a function of time.
>         Data streams can carry one or more data channels where each
>         channel is a single number or a vector.
>
>         An example would be an electrocardiography (ECG) sensor which is
>         used to produce a waveform of the heart's activity over a time
>         period of up to a few seconds (see attached image). In a
>         hospital the sensor and display are connected together with
>         electrical leads.
>
>         In the Web of Things, the sensor and the display are both
>         modelled as "things". In place of leads, we set the sensor's
>         output property to the display. The display could be an object
>         in a web page script that displays the waveform using the HTML
>         CANVAS element, or it could be a remote service, e.g. a device
>         that acts as an oscilloscope. The display could have properties
>         that control its behaviour, e.g. how many seconds of data to
>         show. It could also have actions, e.g. to freeze the display to
>         allow a waveform to be studied in more detail.
>
>         Script developers ony need to know the object model for the
>         sensor and display. The details of the underlying protocols are
>         dealt with automatically by the server platform. For this to
>         work, we need a standard way to describe data streams as part of
>         the data model for "things".
>
>         For a simple thing property, e.g. the state of a light switch,
>         setting the value of the property on the proxy results in the
>         server sending a message to update the physical state of the
>         switch. For streams, the semantics is slightly different.
>         Setting the value of the proxy's' output directs the data stream
>         to the designated object.
>
>         This model assumes that a stream can only be sent to a single
>         data sink.  If you want to have multiple sinks, you will need to
>         use a "thing" that acts as splitter. In principle, the thing
>         acting as the sensor could do this, e.g. providing one output
>         for a data logger, and another for a display.
>
>         p.s. I am planning on implementing this as part of the NodeJS
>         server project on GitHub. By the way, there are several examples
>         of how to create an ECG monitor with Arduino’s if you want to
>         build your own.
>
>         —
>
>             Dave Raggett <dsr@w3.org <mailto:dsr@w3.org>>
>

Received on Wednesday, 3 June 2015 23:42:59 UTC