AW: "topics" in WS-Eventing (was Re: AW: Relationship to OASIS standards)

Hi Gilbert,

 

I read your proposal. It is interesting and shows that WS-Notification seems to be underspecified when it comes to determining which binding is supported by event consumers / sinks. But let us talk about WS-E and your proposal.

 

You said that “one could define a ‘topic’ as ‘a collection of related Notification types which can be subscribed to in a single operation’”. This is not exactly what I am looking for. I am looking for WS-Topic functionality. There (sorry if I repeat stuff that you already know), a topic may define the message types that clients can expect in notifications. However, a topic also has implied semantics which tell the client what the notifications that are posted to a topic will be about. I created the following example of a WS-Topic TopicSet:

 

<?xml version="1.0" encoding="UTF-8"?>

<wstop:TopicSet xmlns:wstop="http://docs.oasis-open.org/wsn/t-1" …>

  <swes:ServiceMetadataChanged wstop:topic="true">

    <OfferingAdded wstop:topic="true"/>

    <OfferingRemoved wstop:topic="true"/>

    <OfferingChanged wstop:topic="true"/>

  </swes:ServiceMetadataChanged>

  <swes:SensorMetadataChanged>

    <SensorRecalibrated wstop:topic="true"/>

  </swes:SensorMetadataChanged>

  <sps:TaskStatusUpdate wstop:topic="true">

    <Accepted wstop:topic="true">

      <igsi:Suspended wstop:topic="true"/>

      <igsi:Resumed wstop:topic="true"/>

    </Accepted>

    <Cancelled wstop:topic="true"/>

    <Failed wstop:topic="true"/>

    <Completed wstop:topic="true"/>

    <NewDataAvailable wstop:topic="true"/>

  </sps:TaskStatusUpdate>

</wstop:TopicSet>

 

A service for tasking sensors might publish notifications on these topics. Whenever the service’s metadata changed, a notification will be generated and sent to subscribed event sinks. If such consumers are only interested in those cases where new metadata was added (OfferingAdded), they could subscribe directly for those messages. Even if all SensorMetadataChanged topics used the same message types, clients could subscribe to specific subsets of these kinds of messages. Having services match new notifications against subscriptions based upon topics also seems to have better performance than doing content based filtering. Well, notifications would need to be published / associated to topics somehow, of course, which requires some extra computing. Btw., in WS-Topics the possible message (schema) types are contained in the topic namespaces used by the service; another construct of WS-Topics that I will skip for the moment. 

So, even if all notifications posted in any of the ServiceMetadataChanged topic branch are about metadata changes and may be of the same message type, the structuring of the sub-topics provide additional meaning to clients. If clients are interested in several topics, they can subscribe to them in one operation request. WS-Topics defines topic filters that can be used to indicate which topics are of interest. So a service can aggregate topics quite flexibly and clients can subscribe for them. If a client does not understand / know a given topic (e.g. all those with prefix ‘sps’ in the example), or are not interested at the moment, they can simply ignore them – either by not including them in their subscription or by dismissing them when they receive it from the service (I prefer the former).

 

WS-Topics allows additional information / documentation to be provided for a topic, both in topic namespaces and in topic sets. This could be used to provide descriptions for humans for direct display in client software or for automatic consumption, like some kind of topic policy. A service might indicate how frequent it will publish on certain topics etc. Right now I do not know of any official usages of these “extension points” but as you see I have some ideas. I am primarily interested in improving the information availability of traditional request-response based service infrastructures (that are common in my domain).

 

Topic sets can also be dynamic, which supports use cases involving brokers or stateful resources managed by a service, like processes.

 

What do you think? Is WS-Topics really such a horrible solution for topic based subscriptions? J I admit that the specification is not an easy read; however, I like the defined functionality. Clients may use only very simple topic constructs (“send me all task status updates for sensor xyz”) or more complex ones (then likely a human is configuring the subscription). Same for services.

 

Your proposal covers one aspect of all this “topic stuff” – to indicate the message types that are used by an event source. In addition, it provides a way to ensure that the event source uses the correct binding when sending notifications to a subscribed event sink (the WS-Notification spec is silent on this aspect, afaik). The additional information I was talking about before could be added to the extension points of WSDL port type operations, not sure about that approach but it seems to be doable. 

However, the approach described in the proposal seems to be quite verbose once you support multiple bindings, multiple sets of notifications, different policies etc. The proposal mentions subscription policies as an alternative, I think this would be a better approach. Maybe I am too biased here because of my previous work with WS-Notification. That has an explicit extension point to add policies that affect a subscription in the Subscribe request used by clients. In conjunction with policies defined by a service and topics you would have a cleaner approach – afaic.

 

Regards,

Johannes

 

 

Von: public-ws-resource-access-request@w3.org [mailto:public-ws-resource-access-request@w3.org] Im Auftrag von Gilbert Pilz
Gesendet: Montag, 20. April 2009 21:01
An: johannes.echterhoff@igsi.eu
Cc: 'Yves Lafon'; public-ws-resource-access@w3.org
Betreff: "topics" in WS-Eventing (was Re: AW: Relationship to OASIS standards)

 

Johannes,

One problem with discussing "topics" is that they aren't very well defined and different technologies (WS-Notification, JMS, etc.) tend to treat them differently. I think some of the requirements you discuss aren't so much about "topics" per se, but rather the basic ability to describe the type/schema of the Notifications. I have submitted a general proposal for advertising Notification types which I've attached to this email. If you could, I would appreciate it if you read it over and let me know what you thought.

Given the ability to describe Notifications types, one could define a "topic" as "a collection of related Notification types which can be subscribed to in a single operation". Does this definition sound reasonable to you, or were you thinking of something else beyond or in addition to this? If such a definition does meet your requirements, then (again, referring to my proposal) It seems to me that a portType within a Notification WSDL is basically the same as a topic.

With regards to Appendix A, "Service Metadata for Eventing", of the current version of WS-Eventing; it has a lot of problems and there is an issue (6661 <http://www.w3.org/Bugs/Public/show_bug.cgi?id=6661> ) filed against it.  I would like to see the proposal for Notification WSDLs adopted and this appendix removed from WS-Eventing.

Finally with regards to the idea of implementing WS-Topics on top of WS-Eventing, I have to confess that my mind recoils with horror :-) I have never been able to understand the underlying requirements for "forests of Topic Trees" or the need for non-centrally controlled, hierarchical, topic inheritance structures, etc. If the use cases in WS-Topics make sense to you, I would like to discuss them with you in the hopes that you can help me understand them.

- gp

Johannes Echterhoff wrote: 

Dear Yves,
 
Thank you very much for your feedback.
 
  

There are no relationship between WS-Eventing work within the WS-RA WG
and
OASIS, unless OASIS TCs want to send issues, in that case they will be
considered like all other issues by the Working Group, and if alignment
is
possible, there is nothing preventing it.
    

 
That is fair enough.
 
  

As those [harmonization] efforts were made outside W3C framework, only participants in that effort are able to comment and explain why it didn't happen.
    

 
Ok - I did not know who exactly was involved. I thought that it was official work, that is why I contacted the working group. I got some feedback from several persons. They could not provide reasons, but at least now I know for sure that no harmonization will happen in the near future.
 
  

Issue is that things that appear to be competing quite often are not. I
wold take the example of WS-Choreography and BPEL, they seem to tackle
the
same issue, but in fact they do it form different angles and are not
really competing (especially as WS-Choreography didn't reach final
REC).
So the right approach is first to settle on the functionnality you want
to
provide and pick the set on specification you want to adhere to based
on
that.
    

 
This is what several different sources told me - and I guess it is the best advice I can get. I will investigate WS-N and WS-E in more detail, especially with respect to their dependent specifications and existing extensions, and see which ultimately best fits the requirements - which I have a good picture of, but they are not written in stone yet.
 
  

There is no "W3C and OASIS architectural styles", both organizations
produced specifications that should help you implement Web Services
using
your architectural style. What matters most is your use cases and what
you
want to achieve.
    

 
What I meant was that there are standards at OASIS and W3C that have the same basic functionality. This is a troubling situation for someone who would like to suggest only one way to do resource management (create,get,modify,delete) and Pub/Sub across web services of a given domain. However, I understand that I will need to make the best of the current situation and find out which way will ultimately best fit the requirements.
 
  

Could you come up with a set of issues based on differences between
your
requirements and the specifications the WS-RA WG is producing? That
would
be the ideal way to understand the issue you have with eventing and
notification.
    

 
Ok, let me try to explain. Basic Pub/Sub functionality (make a subscription, renew it and unsubscribe) is available in both WS-N and WS-E. Both are binding independent or at least claim to be. That's a foundation to build upon.
 
One requirement for me that is not directly supported by WS-RA is the need for topic based subscriptions. Topics are a powerful mechanism to indicate which types of notifications are supported by a service and which are requested by a client. In my domain, the ultimate goal is to have common types of notification messages, possibly with different semantics, and have different service specifications define their additional notification types that are specific to them. Now, when one web service instance implemented several port types defined by different service specifications, it would be great to be able to publish notifications / events that belong to these services on according topics so that on the one hand clients can see which notifications are supported (keeping in mind that generation of some notification types might be optional) and specify exactly (without having to apply content based filtering) which notifications they are interested in. The latter often depends u
pon the client role - e.g. administrative clients will likely be interested or have access to other notifications than end user clients. I think that topics have a lot of value, especially when they indicate meaning to clients and help both services and clients to do Pub/Sub (subscribing using topic based filtering and matching of notifications using topic channels).
 
Something that I recognized lately is that (the current version of) WS-Eventing makes use of notification and solicit-response message exchange patterns (note that I do not know 100% if this is going to change in the final version of WS-Eventing, but I want to mention this anyway). This is good to indicate the exact schema type of messages and possibly faults for different notifications. Topics at least in WS-Notification can - but do not have to - indicate the schema type of notifications published on these topics as well. However, the point is that the WS-I basic profile prohibits use of these message exchange patterns, at least in WSDL 1.1 descriptions. Maybe they did not take into account the work done by WS-Eventing. Maybe I also missed something. What do you think?
 
Back to topic(s): I have been told that this functionality could be added to WS-Eventing. I agree that this is doable - however, one goal for me is to reuse existing functionality as much as possible when it fits the needs. And WS-Topics does that. For WS-Eventing, there is no such mechanism yet. Maybe WS-Topics can be integrated into WS-Eventing. If I would do that (or define my own topic extension), I would do it because the overall mechanism of WS-E as well as the value of its depending standards (like WS-ResourceTransfer) and existing extensions outweigh the use of WS-Notification.
 
In addition to topics, having defined mechanisms to do notification / event brokering as well as on-demand based publishing as defined in WS-Notification is also of high interest to my domain. I would not say that it is a requirement at the moment, where just enabling Pub/Sub functionality in general would be the first step. However, once the transition to a more event-driven architecture is made then being able to add dedicated broker services will likely become a requirement. In fact, we already have use cases where there is one service which acts as an aggregation for several other services of the same type, allowing clients to work with a single endpoint to handle their data / processing requests (the aggregating service would access the underlying services on demand). That service could make good use of a broker that handles all notifications from the underlying services.
 
To summarize: I like the way resource handling is done by WS-(Resource)Transfer, at least how it seems to be compatible with HTTP verbs and thus a RESTful architecture. On the other hand, the functionality provided by WS-Notification fits the requirements that I see at the moment and in the future (basic Pub/Sub, then optionally topics, brokering and on demand based publishing), so is more applicable than WS-Eventing which I would need to extend to fit these needs.
 
Hope that clarified a bit.
 
Cheers,
Johannes
 
 
  

-----Ursprüngliche Nachricht-----
Von: Yves Lafon [mailto:ylafon@w3.org]
Gesendet: Donnerstag, 16. April 2009 19:25
An: Johannes Echterhoff
Cc: public-ws-resource-access@w3.org
Betreff: Re: Relationship to OASIS standards
 
On Mon, 6 Apr 2009, Johannes Echterhoff wrote:
 
Dear Johannes,
 
    

I am new to this list, therefore I am going to quickly introduce
      

myself: my
    

name is Johannes Echterhoff and I am involved in standardization
      

activities
    

of the Open Geospatial Consortium (OGC -
      

http://www.opengeospatial.org).
    

Recently, there was quite a discussion in an OGC working group on how
      

to
    

create standards that support both REST(ful) and WS-* (based upon
      

standards
    

from W3C and OASIS) style web services. A lot of these discussions
      

revolve
    

around the resources we need to model and how to do that in both
architectural styles. For the WS-* style, this brought me to the WS-
      

Resource
    

specifications from OASIS. In addition, we want to integrate
publish/subscribe functionality in some of our web services. This
      

brought me
    

to WS-Notification. I favoured the OASIS specifications because they
      

reached
    

the final specification status, while for example WS-Eventing at the
      

time
    

was "only" a W3C submission.
      

First, the "WS-* style" is not really dictated by all the WS-*
specifications, but mostly by tools, then by selected clusters of
WS-* specification. It is entirely possible to create RESTful Web
Services
using SOAP 1.2 and WSDL, (although a bit difficult to find the tools
for
that), but some WS-* specs are indeed mandating partially the overall
architecture.
 
    

Now that I learnt of the activities of your working group, I have
      

some
    

questions that I hope you can answer:
 
How are the activities of this W3C working group, especially the
      

resulting
    

documents / standards related to the work done by OASIS? For example,
WS-Eventing appears to provide a subset of the functionality given by
WS-Notification.
      

There are no relationship between WS-Eventing work within the WS-RA WG
and
OASIS, unless OASIS TCs want to send issues, in that case they will be
considered like all other issues by the Working Group, and if alignment
is
possible, there is nothing preventing it.
 
    

Some time ago I read about a common approach to eventing in web
      

service
    

environments, titled WS-EventNotification. I thought this was a great
      

idea,
    

having a basic set of functionality which would support basic pub/sub
      

and
    

optional extensions which provided higher functionality, like topic
      

based
    

subscriptions or brokering. Unfortunately, according to
http://travisspencer.com/blog/2008/11/effort-to-converge-ws-
      

eventing.html
    

the efforts to harmonize WS-Eventing and WS-Notification and create a
single, common standard have ceased. Please comment if this is true
      

and if
    

so why the common approach has been abandoned.
      

As those efforts were made outside W3C framework, only participants in
that effort are able to comment and explain why it didn't happen.
 
    

There seems to be functional overlap of WS-(Resource)Transfer with
      

the
    

WS-ResourceProperties specification from OASIS. The current working
      

drafts
    

from your group look like they are more compatible with the http
      

interface
    

(get, put, delete etc.), so the functionality implemented by the
      

proposed
    

operations look like they could be more easily ported to a REST(ful)
      

style
    

web service - is this the intention?
      

WS-Transfer is quite close to the basic definition of verbs used in
HTTP,
WS-ResourceTransfer is one of the possible concrete use of WS-Transfer,
and yes, being closer to HTTP while allowing the same kind of
architectural style running on other underlying transports might indeed
help keeping the same architectural style between both implementations.
 
    

[As for WS-MetadataExchange and WS-Enumeration I do not know of an
      

OASIS
    

equivalent (have heard about WS-MetadataExchange before and found it
      

quite
    

useful), but if there is one, please let me know.]
 
 
For my work at the OGC, I need guidance which approach (that from W3C
      

or
    

OASIS) to use and promote. It would help a lot if there was a public
statement to what extent these apparently competing standards differ
      

or
    

share the same functionality. If possible, guidance why one should
      

use one
    

approach over the other would be highly appreciated - something like
      

best
    

practices for various use cases. Right now the only arguments
      

pro/contra an
    

approach for me are its current specification status
(submission/recommendation/draft/etc.), the functionality
      

required/provided
    

and the available toolbase.
      

Issue is that things that appear to be competing quite often are not. I
wold take the example of WS-Choreography and BPEL, they seem to tackle
the
same issue, but in fact they do it form different angles and are not
really competing (especially as WS-Choreography didn't reach final
REC).
So the right approach is first to settle on the functionnality you want
to
provide and pick the set on specification you want to adhere to based
on
that.
 
    

The thing is that for my OGC related standardization work I need to
      

decide
    

which WS-* standards to use and which not to use. I would not like to
      

end up
    

with a W3C and an OASIS architectural style for the implementation
specification of my geospatial web service, in addition to the
      

REST(ful)
    

style.
      

There is no "W3C and OASIS architectural styles", both organizations
produced specifications that should help you implement Web Services
using
your architectural style. What matters most is your use cases and what
you
want to achieve.
 
    

You see that I am a bit confused with the (apparently emerging)
      

functional
    

overlap of W3C and OASIS specifications. I hope you can help to
      

clarify the
    

current situation (on managing resources and eventing/notification in
      

web
    

services).
      

Could you come up with a set of issues based on differences between
your
requirements and the specifications the WS-RA WG is producing? That
would
be the ideal way to understand the issue you have with eventing and
notification.
Cheers,
 
--
Baroula que barouleras, au tiéu toujou t'entourneras.
 
         ~~Yves