Re: AW: data streams

Hi Claes,

On 07/06/15 02:59, Nilsson, Claes1 wrote:
> Thanks for these links Ari.
>
> The Publish-Subscribe Broker for CoAP was new to me. Sounds interesting and I am reading the Abstract: "This document defines a publish-subscribe broker for CoAP that extends the capabilities of CoAP for supporting nodes with long breaks in connectivity and/or up-time."
>
> I have a two questions on this draft:
>
> * Compared to using MQTT, which are the benefits of the Publish-Subscribe Broker for CoAP? Even more lightweight than MQTT as it runs over UDP?

CoAP is lighter than MQTT, although it's roughly in the same ball park 
with MQTT-SN.

I think the biggest benefit for CoAP pub-sub broker comes from the fact 
that it integrates well with the rest of the CoAP infrastructure, and 
hence works well with the rest of the web (CoAP was built that way).

Also, you can use the CoAP pub-sub broker to facilitate interworking 
between CoAP and MQTT. This part is not specified yet, but it has been 
the idea all along.

> * In the Introduction section issues with middle-boxes, such as NATs and firewalls, and prevention of connecting to a device from the Internet are mentioned. If a client is subscribing to a topic the specification states that it should use a CoAP GET Observe request to the broker. However, when time passes will there not be problems with the Observe messages sent from the broker to the client as we can't expect NATs and firewalls to keep the communication chain open for a long time?

That's true. The initial connectivity problem is solved by 
client-initiated actions, but the keep-alive part needs still needs more 
work. We'll take that to our list of todo-items for the next revision.


Thanks,
Ari

> BR
>    Claes
>
> Claes Nilsson
> Master Engineer - Web Research
> Advanced Application Lab, Technology
>
> Sony Mobile Communications
> Tel: +46 70 55 66 878
> claes1.nilsson@sonymobile.com
>
> sonymobile.com
>
>
>
>> -----Original Message-----
>> From: Ari Keränen [mailto:ari.keranen@ericsson.com]
>> Sent: den 4 juni 2015 01:43
>> To: Hund, Johannes; Dave Raggett; Magnus Olsson; Edoardo Pignotti;
>> public-wot-ig@w3.org
>> Subject: Re: AW: data streams
>>
>> Hi all,
>>
>> I agree that pub-sub model for handling this kind of streams of data
>> makes a lot of sense.
>>
>> For folks not familiar with the IETF work, here's pointers to relevant
>> drafts.
>>
>> Observing Resources in CoAP:
>> https://tools.ietf.org/html/draft-ietf-core-observe-16
>>
>> Publish-Subscribe Broker for CoAP:
>> https://tools.ietf.org/html/draft-koster-core-coap-pubsub-01
>>
>> The observe draft is in final stages to become an RFC while the pub-sub
>> broker draft is still in early stages of specification.
>>
>> I'm one of the co-authors of the pub-sub broker draft and we are
>> currently updating it for the next IETF meeting. Comments on that are
>> very welcome.
>>
>>
>> Cheers,
>> Ari
>>
>>
>> On 03/06/15 06:12, Hund, Johannes wrote:
>>> Hi Magnus,
>>>
>>> in the discussions @IETF there are basically two models being
>>> discussed, even there the discussion is quite related  with the used
>>> protocol (CoAP), the concepts are remarkable generic.
>>>
>>> There one Architecture model is having a central pub-sub broker (can
>>> also be “central”  for a small piece of the application such as a
>> room
>>> or a home), while the other is more generalized to have a way to
>>> orchestrate data bindings, i.e. to tell one resource to convey data
>>> changes to or from another resource. That means I could tell one
>>> resource to observe another resource (and optionally tell it to
>> change
>>> its state accordingly) or I can tell one resource to push any data
>>> changes to another resource. The pub-sub broker would be a special
>>> case of that.
>>>
>>>   From the current discussions of the WoT framework e.g. between Vlad
>>> and me, I would see this also covered by the current approach, as one
>>> “thing” can have properties and you should be able to subscribe to a
>>> property to get informed on data change.  How this subscription is
>>> done is dependent on the protocol that the framework resp. model is
>> mapped to.
>>>
>>> Same goes with events, the main difference between a property change
>>> and an event is that the event might have a greater semantic meaning
>>> e.g. a flatline of the ECG or e.g. might be referring to several
>>> properties, such as heart rate above a threshold value.
>>>
>>> I really think that the model we currently are discussing is capable
>>> of providing these use cases Dave mentioned, but we should define
>>> higher-level patterns (e.g. sensor value  + threshold as properties
>>> and an event source for threshold traversal or denying subscriptions
>>> if there is already one active subscription) based on the framework
>>> that could e.g. have own APIs or be described as such in the TDL.
>> That
>>> is for example how the CoRE interfaces are done in the IETF, they are
>>> basically interaction patterns of a RESTful API.
>>>
>>> I was not quite sure in the discussion below: is there the suggestion
>>> to forsee a way to manage out-of-band streams or is it referring to
>> inband?
>>>
>>> Also it might be useful, I am not sure if we should address
>> management
>>> of out-of-band streams, as it opens quite a can of worms. If the
>>> consensus is we do need it, then I would see it sufficient to also
>>> provide a sort of pattern as mentioned above.
>>>
>>> Any thoughts?
>>>
>>> Best Regards,
>>>
>>> Johannes
>>>
>>> *Von:*Magnus Olsson [mailto:magnus.olsson@ericsson.com]
>>> *Gesendet:* Mittwoch, 3. Juni 2015 10:27
>>> *An:* Edoardo Pignotti; public-wot-ig@w3.org
>>> *Betreff:* Re: data streams
>>>
>>> Edoardo,
>>>
>>> Great if it works I have no practical experience myself using the
>>> pub-sub for IoT data but understand that there is standards work (in
>>> IETF) that could be possible to reference. What is the Oxford flood
>>> network, sounds like an Iot mesh?
>>>
>>> Br magnus
>>>
>>> http://www.ericsson.com/current_campaign
>>> <http://www.ericsson.com/current_campaign>
>>>
>>> *MAGNUS OLSSON *
>>>
>>>
>>> Ericsson AB
>>> Group Function Technology
>>> Mobilvägen 1
>>> 22362 Lund, Sweden
>>> SMS/MMS +46 706 756814
>>> magnus.olsson@ericsson.com <mailto:magnus.olsson@ericsson.com>
>>> www.ericsson.com <http://www.ericsson.com>
>>>
>>>
>>>
>>> Ericsson
>>>
>>>
>>>
>>> This Communication is Confidential. We only send and receive email on
>>> the basis of the terms set out at www.ericsson.com/email_disclaimer
>>> <http://www.ericsson.com/email_disclaimer>
>>>
>>> On 2015-06-03 10:19, Edoardo Pignotti wrote:
>>>
>>>      Hello,
>>>
>>>      I also agree that streams would be a very useful thing to support
>> in
>>>      the WoT framework.
>>>
>>>      I am in favour of a pub-sub broker model. In the oxford flood
>>>      network we thought very hard how to manage data streams and we
>> went
>>>      for a pub-sub model.
>>>
>>>      I would be more than happy to discuss this further in the context
>> of
>>>      our case-study.
>>>
>>>      All the best,
>>>
>>>      Edoardo
>>>
>>>      *From: *Magnus Olsson <magnus.olsson@ericsson.com
>>>      <mailto:magnus.olsson@ericsson.com>>
>>>      *Organization: *Ericsson AB
>>>      *Date: *Wednesday, 3 June 2015 09:04
>>>      *To: *"public-wot-ig@w3.org <mailto:public-wot-ig@w3.org>"
>>>      <public-wot-ig@w3.org <mailto:public-wot-ig@w3.org>>
>>>      *Subject: *Re: data streams
>>>      *Resent-From: *<public-wot-ig@w3.org <mailto:public-wot-
>> ig@w3.org>>
>>>      *Resent-Date: *Wednesday, 3 June 2015 09:05
>>>
>>>      Dave,    (and woters)
>>>
>>>      Streams sounds like a very useful thing to support :-)
>>>
>>>      - For the multiple sinks, why not use a pub sub broker as the
>>>      identified source of the stream that way any number of sinks can
>>>      attach to that pub-source. Seems to me that setting a proxy
>> output
>>>      value to point to a single output is a limitation that might
>> result
>>>      in someone creating an upstream proxy to split to multiple sinks.
>>>
>>>      Any thing should be able to redirect any aspect of their source
>>>      (defined by its meta description) to a nearby pub-sub proxy
>> service.
>>>      Still that could be an "on device" (localhost) broker, a home
>>>      broker, a service provider broker or a public broker etc.
>>>
>>>      It seems reasonable to me to have use case such as one part of
>> the
>>>      stream for the snapshot, another for the live view a third for
>> data
>>>      analysis, a fourth for a remote tele medicine, a fifth for a
>>>      emergency monitoring function etc.
>>>
>>>      - Even discrete value type of sensors could be handled as a
>> stream
>>>      if only given long enough time to observe
>>>
>>>      - (Not convinced myself) but perhaps any data points should be
>>>      allowed to be retrieved as a stream with sometime only a single
>>>      value being sent before the connection is being terminated.
>>>
>>>      - If there is a possibility to "cascade" (or chaining) the
>> streamed
>>>      data points then it might open up for a "network of" data stream
>>>      functions that could apply smartness to the stream such as adding
>>>      timestamps (simple sensors do not have timestamps, but sometimes
>>>      have sample rates), compression, average, de-noising, composite
>> streams.
>>>
>>>      Thanks!
>>>
>>>
>>>      Br magnus
>>>
>>>      http://www.ericsson.com/current_campaign
>>>      <http://www.ericsson.com/current_campaign>
>>>
>>>      *MAGNUS OLSSON *
>>>
>>>
>>>      Ericsson AB
>>>      Group Function Technology
>>>      Mobilvägen 1
>>>      22362 Lund, Sweden
>>>      SMS/MMS +46 706 756814
>>>      magnus.olsson@ericsson.com <mailto:magnus.olsson@ericsson.com>
>>>      www.ericsson.com <http://www.ericsson.com>
>>>
>>>
>>>
>>>      Ericsson
>>>
>>>
>>>
>>>      This Communication is Confidential. We only send and receive
>> email
>>>      on the basis of the terms set out at
>>>      www.ericsson.com/email_disclaimer
>>>      <http://www.ericsson.com/email_disclaimer>
>>>
>>>      On 2015-06-03 09:16, Dave Raggett wrote:
>>>
>>>          I am interested in enabling the Web of Things Framework to
>>>          handle data streams.
>>>
>>>          Some sensors produces streams of data as a function of time.
>>>          Data streams can carry one or more data channels where each
>>>          channel is a single number or a vector.
>>>
>>>          An example would be an electrocardiography (ECG) sensor which
>> is
>>>          used to produce a waveform of the heart's activity over a
>> time
>>>          period of up to a few seconds (see attached image). In a
>>>          hospital the sensor and display are connected together with
>>>          electrical leads.
>>>
>>>          In the Web of Things, the sensor and the display are both
>>>          modelled as "things". In place of leads, we set the sensor's
>>>          output property to the display. The display could be an
>> object
>>>          in a web page script that displays the waveform using the
>> HTML
>>>          CANVAS element, or it could be a remote service, e.g. a
>> device
>>>          that acts as an oscilloscope. The display could have
>> properties
>>>          that control its behaviour, e.g. how many seconds of data to
>>>          show. It could also have actions, e.g. to freeze the display
>> to
>>>          allow a waveform to be studied in more detail.
>>>
>>>          Script developers ony need to know the object model for the
>>>          sensor and display. The details of the underlying protocols
>> are
>>>          dealt with automatically by the server platform. For this to
>>>          work, we need a standard way to describe data streams as part
>> of
>>>          the data model for "things".
>>>
>>>          For a simple thing property, e.g. the state of a light switch,
>>>          setting the value of the property on the proxy results in the
>>>          server sending a message to update the physical state of the
>>>          switch. For streams, the semantics is slightly different.
>>>          Setting the value of the proxy's' output directs the data
>> stream
>>>          to the designated object.
>>>
>>>          This model assumes that a stream can only be sent to a single
>>>          data sink.  If you want to have multiple sinks, you will need
>> to
>>>          use a "thing" that acts as splitter. In principle, the thing
>>>          acting as the sensor could do this, e.g. providing one output
>>>          for a data logger, and another for a display.
>>>
>>>          p.s. I am planning on implementing this as part of the NodeJS
>>>          server project on GitHub. By the way, there are several
>> examples
>>>          of how to create an ECG monitor with Arduino’s if you want to
>>>          build your own.
>>>
>>>          —
>>>
>>>              Dave Raggett <dsr@w3.org <mailto:dsr@w3.org>>
>>>
>>
>

Received on Monday, 8 June 2015 23:34:12 UTC