Re: Transfer layer abstaction model with notifications

> On 2 May 2016, at 23:52, Michael Koster <michael.koster@smartthings.com> wrote:
> 
> Here is a presentation that describes a proposal for a simple abstract transfer layer that can be used to map TD items (events, actions, properties) to resources using CoAP, HTTP, and MQTT bound protocols. 
> 
> This includes a consistent transfer layer abstraction for asynchronous notifications from actions, events and resource state changes using existing mechanisms for CoAP (Observe), HTTP (EventSource), and MQTT (pubsub). 
> 
> https://github.com/connectIOT/Architecture/blob/master/wot-2may16.pdf <https://github.com/connectIOT/Architecture/blob/master/wot-2may16.pdf>
> 
> Best regards,
> 
> Michael
> 
> PS I also propose a "group" TD item extension type that is optional and can be used to expose collections of resources for collective interaction. It could be a starting point for defining other types of collections. 
> 
> PPS I think Actions, Events, and Properties are also fundamentally collections. Ultimately a collection tag like "Actions" or "Properties", can be mapped to a common link relation "hasAction" or "hasProperty" wrt. the context of the "thing" being described. This mapping can be automated using JSON-LD processing…

Thanks Michael for sharing this. I am trying to align your ideas with mine.

My starting point is the assumption that the interfaces exposed by applications should be decoupled from the communication patterns and protocol bindings. This ensures that application code isn’t tied to the details of the protocols. Application designers should be able to influence the lower layers via communications metadata, and through the choice of platforms for their services. Given this, the subscription mechanism for applications should be considered independently from subscription mechanisms used with specific protocols. 

I have found it helpful to think about abstract messages independently of the underlying protocols. We only need a few kinds of abstract messages to support the application interfaces. These include:

   a.  register/unregister a local proxy for a remote thing
   b.  register/unregister a remote proxy for a local thing
   c.  signal the death of a thing or ancestor proxies
   d.  property update
   e.  event
   f.  invoke action
   g.  response to an action
   h. subscribe/unsubscribe to events

When a thing goes away you need to inform its proxies so that they in turn can notify the applications on each server. I allow for a tree of proxies rooted with a thing. This implies the need to deal with the case for the death of a proxy that is not a leaf node in the tree. The simple approach is to unregister the proxy with its parent, and notify all of the proxy's children (and in turn their children recursively). Applications can then choose to try another way to recover.

The property update could be initiated by the thing in which case the update needs to be communicated to the proxies. The update could also be initiated by one of the proxies, in which case the update needs to be communicated to the thing at the root of the proxy tree as well as to all of the proxies in the tree. Proxy registration is essentially equivalent to your “observe” in respect to property updates.

Actions may have a sequence of responses. The abstract messages thus need an identifier that matches a response to the specific action invocation. JSON RPC provides an example for how this can be realised. The scope of the identifier ends with the last response or a timeout.  The action invocation is equivalent to your “observe” operation in respect to action responses.

I’ve included abstract messages for subscribing and unsubscribing to event streams. This is equivalent to your “observe” operation in respect to event notifications.  If allows nodes in the proxy tree to determine whether or not to forward event notifications, either to the root of the tree or toward the leaves.

The abstract messages are mapped to specific communication patterns according to the thing description’s communication metadata. In some cases, we may use polling with a pull protocol, in others, there may be a way handle asynchronous notifications directly.  In the Compose project, for example, the platform exposed URIs for clients to push notifications to the cloud.

Pub/sub protocols like MQTT and XMPP make it easy for an IoT device to broadcast notifications to all of the clients that have registered an interest.  However, it is harder with these protocols to implement actions where a specific client invokes an action on the IoT device, and the response needs to be sent only to that client.  One solution is to have a topic name specific to a thing that is used to broadcast notifications, a topic name specific to a thing for sending messages to it alone, and a topic name specific to a client that the thing can use to route action responses to just that client.

We also need to allow for buffering and multiplexing of notifications. In some cases, there is a need to buffer notifications and then send a block of them. Likewise, we may want to combine notifications from multiple sensors into the same block. The communications metadata enables platforms to perform the appropriate mapping to and from abstract messages.

In practice, abstract messages are entries in queues held by the platform. When a script updates a property value, this results in pushing the update to a queue. In my open source POSIX/C++ implementation, I am using the Unix “select" system call to manage the server's IP sockets. When a socket becomes available for writing and there is a message to be sent, a state machine is updated to incrementally encode the message and write it to the socket.  When a socket has something to read on it, a state machine is updated to incrementally decode the message, and when it is complete to push it to a queue for delivery to the things/proxies it is targeted at.

I am working on support for multiple protocols and representations with a view to exploring the practical implications for richer type system for thing data models, and for integrating different platforms as part of the web of things. This involves a lot of work and will take time to see through.

Best regards,
—
   Dave Raggett <dsr@w3.org <mailto:dsr@w3.org>>