Re: [GitHub] Organizing the repository

Hi Johannes,

I have been thinking about the difference being that in one case, the interaction is between the resource layer and an application layer, and in the other case the interaction is contained within the resource layer, between resources. 

Using explicit resources for each case is a good idea, and while it doesn't reduce the resource requirements per connection, it does make the connections visible and manageable.

The motivation is to re-use the server architecture for notifications, where there is a request submitted that may carry an implicit or explicit subscription; CoAP Observe, EventSource, MQTT Subscribe. The servers are already "plumbed" for these and it is a substantial addition to put in a RESThook that doesn't have an associated request parameter block with things like media type described. I ended up deciding to re-use the existing framework with external bindings rather than add a new server module.

To make these visible and manageable, we could create resources that only use the server pattern locally, i.e. implement a binding resource that generates a local Observe request to the resource. For each binidng, there would be exactly one RESThook that uses the long-lived request locally, and the binding could be to another observable resource or to a broker. 

In some simple systems, it would make sense to observe resources directly. I didn't want to prohibit direct use of CoAP Observe or SSE in cases where it makes sense. For fan-out cases, one could create a binding to another observable resource in a less constrained node or a broker, which could itself be observed more broadly, as with a MQTT broker. 

Do you think we should avoid using CoAP Observe or SSE entirely? This introduces the new problem of how to create a portable method of expressing function pointers to callbacks in bindings. If we assume that a protocol has a well known application layer callback method, we can build a general one into the server and use it for resource layer bindings as well.

If we separate the concepts of binding and observing, we can allow efficient configurations for both local and remote asynchronous notifications and state updates. 

I'll update the slides to indicate local bindings used to manage observers on a resoure as the preferred pattern. 

Best regards,

Michael

> On May 2, 2016, at 11:36 PM, Hund, Johannes <johannes.hund@siemens.com> wrote:
> 
> Hey Michael,
>  
> I’ve been thinking exactly about the same topic - the conclusion was a bit different for me:
> I agree that on a protocol level (i.e. for mapping) it makes sense to differentiate if the receiver of notfications keeps an open channel (case 1) or just registers und unregisters a callback (case 2).
>  
> For an application however, the difference is very subtle – there is something where it needs to get updated about and it gets updates.
>  
> Regarding the WoT architecture, this difference could show up on the resource layer, where it is natural to have a receiving resource for “case 2”, but might feel artificial to create resources for “case 1” – I still would recommend doing so, and here is why: with Observe, EventSource, GET-Upgrade, MQTT sub etc.  you have some hidden state e.g. how many subscriptions are already there.
> This is in any practical setup little issue in desktop and cloud environments (except for extreme fan-outs), but can have important impact on constrained devices. 
> Using something like subscription resources also for this case can make that visible/debuggable/manageable, plus some nice features like e.g. most protocols tie that state to the connection, enabling a one-stop view of connected subscribers etc.
>  
> Best regards,
> Johannes
>   <>
> Von: Michael Koster [mailto:michael.koster@smartthings.com] 
> Gesendet: Montag, 2. Mai 2016 15:31
> An: Hund, Johannes (CT RDA NEC EMB-DE)
> Cc: Kovatsch, Matthias (CT RDA NEC EMB-DE); Charpenay, Victor (CT RDA NEC WOS-DE); Public Web of Things IG; Soumya Kanti Datta; louay.bassbouss@fokus.fraunhofer.de; tkamiya@us.fujitsu.com
> Betreff: Re: [GitHub] Organizing the repository
>  
> Great, here is some more discussion.
>  
> I agree that events are different from state changes of properties, and I think they can use some common system design patterns. and have. Subscription is a useful term for the process of registering intent to recieve notifications, but may be overloaded already. We also use publish/subscribe interfaces like MQTT which also have subscription methods.
>  
> We could define "Subscription" in a consistent way for both Events and Properties as the process of registering for notifications, and use the asynchronous communication pattern of "Notification" for both events and property state change propagation. 
>  
> But there are 2 kinds of subscription; one is like CoAP Observe, HTTP EventSource or MQTT subscribe, where a program makes a request and provides a handle for asynchronous callbacks. The second kind where the state of a resource is observed by a monitor and pushed to another resource (URI). This can modify another resource state but is more difficult to connect to a callback handler. The pushed-to resource then needs to be observed in order to generate program callbacks.
>  
> Both events and state changes generate notifications and can be subscribed to, but the behavior has some subtle differences:
>  
> Case 1:  Software application does an extended read operation and receives notifications (CoAP Observe, HTTP SSE, MQTT Subscribe operation)
> Case 2: A control is created to monitor a specific resource and send state updates to another resource (CoRE Interfaces Link Binding, WoT Subscription resource)
>  
> I think we need two different terms for these, and I have been using "Observe operation" for the first case and "Binding" for the second case. I leave "subscription" to describe the pub/sub case only, even though it is similar to Observe.
>  
> Likewise a state update in Case 1 is a response to a particular operation on the resource. A state update in Case 2 is an unsolicited transaction. I have been calling Case 1 "Notification and Case 2 "Publishing"
>  
> Also, when we invoke an action, there is an observe/notify relationship with the new actuation resource which can reuse either of these patterns, but most likely to use Case 1. 
>  
> In the implementation, it has been useful to make one method for registering with a resource that is used by both program observation and bindings.
>  
> I am preparing a contribution as a follow on to my earier article to propopse a consistent protocol for these operations, and the most difficult part seems to be the nomenclature.
>  
> Best regards,
>  
> MIchael
>  
>  
> On May 2, 2016, at 5:46 AM, Hund, Johannes <johannes.hund@siemens.com <mailto:johannes.hund@siemens.com>> wrote:
>  
> Yes, we could do that.
>  
> That write-down was born out of the need to have something on resource-layer to enable protocol-agnostic subscriptions for events and/or bindings or however we might call those two types, so it might be good to have the neutral term.
>  
> BR
>  
> Von: Kovatsch, Matthias (CT RDA NEC EMB-DE) 
> Gesendet: Montag, 2. Mai 2016 13:55
> An: Charpenay, Victor (CT RDA NEC WOS-DE); public-wot-ig@w3.org <mailto:public-wot-ig@w3.org>; michael.koster@smartthings.com <mailto:michael.koster@smartthings.com>; Hund, Johannes (CT RDA NEC EMB-DE)
> Cc: Soumya-Kanti.Datta@eurecom.fr <mailto:Soumya-Kanti.Datta@eurecom.fr>; louay.bassbouss@fokus.fraunhofer.de <mailto:louay.bassbouss@fokus.fraunhofer.de>; tkamiya@us.fujitsu.com <mailto:tkamiya@us.fujitsu.com>
> Betreff: RE: AW: [GitHub] Organizing the repository
>  
> Indeed we need to clarify the topic "events". Carsten Bormann had a nice writeup of the two distinct flavors.
> 
> Should we collect the discussion/survey/conclusion in your spin-off activity? Then I would leave the more neutral term "subscriptions" for now. 
> 
> Ciao
> Matthias 
> 
> 
> Sent from my Android phone using Symantec TouchDown (www.symantec.com <http://www.symantec.com/>)
> 
> -----Original Message----- 
> From: Hund, Johannes (CT RDA NEC EMB-DE) [johannes.hund@siemens.com <mailto:johannes.hund@siemens.com>]
> Received: Monday, 02 May 2016, 12:30
> To: Kovatsch, Matthias (CT RDA NEC EMB-DE) [matthias.kovatsch@siemens.com <mailto:matthias.kovatsch@siemens.com>]; Charpenay, Victor (CT RDA NEC WOS-DE) [victor.charpenay@siemens.com <mailto:victor.charpenay@siemens.com>]; michael.koster@smartthings.com <mailto:michael.koster@smartthings.com> [michael.koster@smartthings.com <mailto:michael.koster@smartthings.com>]; public-wot-ig@w3.org <mailto:public-wot-ig@w3.org> [public-wot-ig@w3.org <mailto:public-wot-ig@w3.org>]
> CC: Soumya Kanti Datta [Soumya-Kanti.Datta@eurecom.fr <mailto:Soumya-Kanti.Datta@eurecom.fr>]; Bassbouss, Louay [louay.bassbouss@fokus.fraunhofer.de <mailto:louay.bassbouss@fokus.fraunhofer.de>]; Takuki Kamiya [tkamiya@us.fujitsu.com <mailto:tkamiya@us.fujitsu.com>]
> Subject: AW: [GitHub] Organizing the repository
> 
> > @Johannes: I will move your temporary item to https://github.com/w3c/wot/tree/master/proposals/subscriptions <https://github.com/w3c/wot/tree/master/proposals/subscriptions> -- unless this is the basis for our events design pattern, in which case I would call it https://github.com/w3c/wot/tree/master/proposals/events <https://github.com/w3c/wot/tree/master/proposals/events> :)
> 
> Perfect thanks.
> Regarding the name:
> Names are "seach-replace", but I think we should pick up this discussion and have two distinct concepts and glossary terms for "state changes as updates to sync resources" and "distinct events", whatever the names are then.
> 
> BR Johannes