Re: [tc-push-l] WebDAV PUSH based on RFC 6202

Hi all,

today, "AwareDAV" has been brought to my attention, see 
http://www2005.org/docs/p180.pdf
Does anyone have any experience with it?

cheers

Marten


Am 16.04.2014 16:22, schrieb Jesse Thompson:
> I chimed in on this conversation on thetc-push-l@lists.calconnect.org  
> list.  Cyrus asked me to post my comments here.  (I tried to refactor
> content from multiple messages for context, but I apologize if it is
> still a bit jumbled - and for top posting)
>
> I don't have much knowledge in implementing push/realtime HTTP other
> than installing various web clients on top of Ejabberd using XEP-0206
> (XMPP over BOSH) and deciding not to use XEP-0025 (Jabber XMPP Polling
> at UW-Madison.
> http://xmpp.org/extensions/xep-0206.html
> http://xmpp.org/extensions/xep-0025.html
>
> Otherwise, what I am saying is based on what I have read or what others
> have told me, so take it with a grain of salt.
>
> I suggested looking at the work being done on websockets for XMPP.  Not
> that XMPP should be used as the protocol, but rather that they might
> have documented some of the same hurdles you are trying to overcome.
> http://tools.ietf.org/html/draft-ietf-xmpp-websocket-02
>
>   From my standpoint as a calendar server administrator, I shudder to
> think of the impact of having tens of thousands of clients keeping
> connections open to my main Apache preforking servers.  The root of my
> recommendation is not to say whether long polling is better or worse
> than web sockets, but rather to suggest the option to have the clients
> make these long running requests to a secondary URI.  This would allow
> server administrators to handle those request with a web server
> specifically optimized for that type of connection.
>
> In response to Brad, who brought up the topic of what data is sent over
> the websocket, I said that I would consider it an implementation detail
> as to what information (and how much) to send over the web socket
> connection vs. telling the client to sync via the normal connection.
>
> The people I know who have implemented realtime web apps generally ditch
> the idea of long polling because it doesn't scale.  They use web
> sockets, but they have clients connect on a different connection than
> the main web server that is serving normal requests.
>
> Not all web servers are non-blocking/asynchronous, so you might not be
> able/willing to completely replace existing web servers and downstream
> reverse proxy servers.  The 2-web server strategy allows you to add in
> push capabilities while remaining backwards compatible, as well as
> segregate load.  If your web sockets server becomes overloaded, clients
> should be able to fall back on the non-push capabilities.
>
> For an example, I know the guys that built
> https://www.thegamecrafter.com/   Their main web servers are not handling
> any of the web sockets connections.  But their application is still
> capable of pushing real time updates.  They accomplish this by having
> the client (javascript) maintain a connection to
> https://www.firebase.com/  to receive those updates.
>
> Firebase is an example of a cloud hosting service for realtime apps.  It
> allows you to outsource the realtime/push capabilities of your apps.  I
> don't know if something like that would be appropriate for calendaring,
> but it might be an instructive exercise to proof-of-concept leveraging
> this type of solution for your work on tc-push.  It might at least lead
> you to the answers to the questions your were asking.
>
> Anyway, I hope this is helpful information.
>
> --
> Jesse Thompson
> Technical Consultant - Messaging
> Productivity and Collaboration Solutions
> Division of Information Technology
> University of Wisconsin–Madison
>
> On 4/9/2014 11:28 AM, Ken Murchison wrote:
>> All,
>>
>> The Calendaring and Scheduling Consortium (CalConnect) is looking at
>> ways to have a server "push" changes made to a calendar/addressbook
>> collection out to a client.  There are already a few proprietary
>> mechanisms in place for doing so, but we would like to come up with
>> something standard that would be relatively simple to implement for both
>> clients and servers and would be applicable to any DAV collection.
>>
>> One idea that we are toying with is to leverage the existing
>> DAV:sync-collection REPORT<http://datatracker.ietf.org/doc/rfc6578/>
>> and HTTP long-polling<http://datatracker.ietf.org/doc/rfc6202/>.  I
>> spent a few hours coding up a prototype version of HTTP long-polling and
>> HTTP streaming for DAV:sync-collection REPORTs in my server which I
>> describe below.  We (CalConnect) are considering using this approach or
>> something similar as a starting point and we are interested in any/all
>> feedback from the larger DAV community, including:
>>
>>    * Is this approach sane, is there a better way, or is any type of push
>>      via HTTP a hopeless endeavor?
>>    * Will this approach (or anything similar) break in the face of
>>      intermediaries?
>>    * Will existing HTTP/DAV stacks be able to handle long-polling and/or
>>      streaming?
>>    * Should the server advertise its ability to long-poll and/or stream
>>      for the client to discover or simply leave it up to the client try
>>      one or both and see what the server does, as is the case in my
>>      implementation below?
>>
>>
>> Long-polling:
>>
>> For long-polling, I leveraged the HTTP Prefer header
>> <http://tools.ietf.org/html/draft-snell-http-prefer>  and its 'wait'
>> preference as a way for the client to tell the server that it wants to
>> long poll.  If the client doesn't specify a DAV:sync-token (initial
>> sync) or if there have been changes since the specified token, then the
>> server will respond immediately.  Otherwise, it will only respond if a
>> change is detected or when the timeout expires, whichever comes first.
>> In the case of a delayed response, I issue a 100 (Continue) provisional
>> response with a Preference-Applied header to notify the client that the
>> server is indeed long-polling as requested.  This provisional response
>> may or may not be necessary.  In my implementation I wait 1 sec less
>> than specified to account for processing time so I don't go over what
>> the client expects.
>>
>> Streaming:
>>
>> The client can request streaming behavior by simply including an Accept
>> header with the 'multipart/mixed' media type (I chose this subtype for
>> lack of something better - we could use the existing x-mixed-replace or
>> create our own).  The client can also specify a timeout for the
>> streaming using the same 'wait' preference as used for long-polling.  In
>> the absence of a client-requested timeout, the server will continue to
>> add body parts until the client disconnects or the server hits some
>> internal timeout.  Because a multipart response allows for an epilogue
>> following the final delimiter, a client can't just rely on the
>> delimiters to detect the end of the response.  Therefore, the server
>> MUST use either chunked TE or close the connection following the
>> multipart response.  In my example below, I close-delimit the response
>> for better readability.  FWIW, I think the same holds true for
>> multipart/byteranges responses.  In my implementation I include a
>> Content-Length header in the body-part headers so the client can detect
>> the end of the XML body without looking for the trailing delimiter
>> (mainly because the trailing delimiter doesn't appear until the next
>> body part).
>>
>> Examples:
>>
>> Here is an example of long-polling with 3 requests (I added superfluous
>> Date headers to show the timing).  The first request returns immediately
>> due to a pre-existing change, the second returns upon detecting a
>> subsequent change some 95 sec later, and the third times out after 3 min.
>>
>> REPORT /dav/calendars/user/ken/Default/ HTTP/1.1
>> Host: localhost
>> Date: Wed, 30 Oct 2013 18:11:11 GMT
>> Content-Type: application/xml
>> Content-Length: 260
>> Prefer: return=minimal, wait=180
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <C:sync-collection xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
>> <D:sync-token>http://cyrusimap.org/ns/sync/1368011844-354</D:sync-token>
>>     <D:sync-level>1</D:sync-level>
>>     <D:prop/>
>> </C:sync-collection>
>>
>> HTTP/1.1 207 Multi-Status
>> Date: Wed, 30 Oct 2013 18:11:11 GMT
>> Vary: Accept-Encoding, Brief, Prefer
>> Preference-Applied: return=minimal, wait=180
>> Content-Type: application/xml; charset=utf-8
>> Content-Length: 421
>>
>> <?xml version="1.0" encoding="utf-8"?>
>> <D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
>>     <D:response>
>> <D:href>/dav/calendars/user/ken/Default/4E4B3490-6F01-41B9-AA5B-FE2CD6A30632.ics</D:href>
>>       <D:propstat>
>>         <D:prop/>
>>         <D:status>HTTP/1.1 200 OK</D:status>
>>       </D:propstat>
>>     </D:response>
>> <D:sync-token>http://cyrusimap.org/ns/sync/1368011844-355</D:sync-token>
>> </D:multistatus>
>>
>>
>>
>> REPORT /dav/calendars/user/ken/Default/ HTTP/1.1
>> Host: localhost
>> Date: Wed, 30 Oct 2013 18:11:12 GMT
>> Content-Type: application/xml
>> Content-Length: 260
>> Prefer: return=minimal, wait=180
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <C:sync-collection xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
>> <D:sync-token>http://cyrusimap.org/ns/sync/1368011844-355</D:sync-token>
>>     <D:sync-level>1</D:sync-level>
>>     <D:prop/>
>> </C:sync-collection>
>>
>> HTTP/1.1 100 Continue
>> Date: Wed, 30 Oct 2013 18:11:12 GMT
>> Preference-Applied: wait=180
>>
>> HTTP/1.1 207 Multi-Status
>> Date: Wed, 30 Oct 2013 18:12:47 GMT
>> Vary: Accept-Encoding, Brief, Prefer
>> Preference-Applied: return=minimal, wait=180
>> Content-Type: application/xml; charset=utf-8
>> Content-Length: 375
>>
>> <?xml version="1.0" encoding="utf-8"?>
>> <D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
>>     <D:response>
>> <D:href>/dav/calendars/user/ken/Default/4E4B3490-6F01-41B9-AA5B-FE2CD6A30632.ics</D:href>
>>       <D:status>HTTP/1.1 404 Not Found</D:status>
>>     </D:response>
>> <D:sync-token>http://cyrusimap.org/ns/sync/1368011844-356</D:sync-token>
>> </D:multistatus>
>>
>>
>>
>> REPORT /dav/calendars/user/ken/Default/ HTTP/1.1
>> Host: localhost
>> Date: Wed, 30 Oct 2013 18:12:48 GMT
>> Content-Type: application/xml
>> Content-Length: 260
>> Prefer: return=minimal, wait=180
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <C:sync-collection xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
>> <D:sync-token>http://cyrusimap.org/ns/sync/1368011844-356</D:sync-token>
>>     <D:sync-level>1</D:sync-level>
>>     <D:prop/>
>> </C:sync-collection>
>>
>> HTTP/1.1 100 Continue
>> Date: Wed, 30 Oct 2013 18:12:48 GMT
>> Preference-Applied: wait=180
>>
>> HTTP/1.1 207 Multi-Status
>> Date: Wed, 30 Oct 2013 18:15:47 GMT
>> Vary: Accept-Encoding, Brief, Prefer
>> Preference-Applied: return=minimal, wait=180
>> Content-Type: application/xml; charset=utf-8
>> Content-Length: 202
>>
>> <?xml version="1.0" encoding="utf-8"?>
>> <D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
>> <D:sync-token>http://cyrusimap.org/ns/sync/1368011844-356</D:sync-token>
>> </D:multistatus>
>>
>>
>>
>> Here is the same sequence of events utilizing streaming with a timeout:
>>
>> REPORT /dav/calendars/user/ken/Default/ HTTP/1.1
>> Host: localhost
>> Date: Wed, 30 Oct 2013 18:11:06 GMT
>> Content-Type: application/xml
>> Content-Length: 260
>> Prefer: return=minimal, wait=180
>> Accept: multipart/mixed
>>
>> <?xml version="1.0" encoding="UTF-8"?>
>> <C:sync-collection xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
>> <D:sync-token>http://cyrusimap.org/ns/sync/1368011844-354</D:sync-token>
>>     <D:sync-level>1</D:sync-level>
>>     <D:prop/>
>> </C:sync-collection>
>>
>> HTTP/1.1 207 Multi-Status
>> Connection: close
>> Date: Wed, 30 Oct 2013 18:11:06 GMT
>> Vary: Accept-Encoding, Brief, Prefer
>> Preference-Applied: return=minimal, wait=180
>> Content-Type: multipart/mixed;
>> boundary="localhost-29378-1383156666-1025603243"
>>
>> This is a message with multiple parts in MIME format.
>>
>> --localhost-29378-1383156666-1025603243
>> Date: Wed, 30 Oct 2013 18:11:06 GMT
>> Content-Type: application/xml; charset=utf-8
>> Content-Length: 421
>>
>> <?xml version="1.0" encoding="utf-8"?>
>> <D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
>>     <D:response>
>> <D:href>/dav/calendars/user/ken/Default/4E4B3490-6F01-41B9-AA5B-FE2CD6A30632.ics</D:href>
>>       <D:propstat>
>>         <D:prop/>
>>         <D:status>HTTP/1.1 200 OK</D:status>
>>       </D:propstat>
>>     </D:response>
>> <D:sync-token>http://cyrusimap.org/ns/sync/1368011844-355</D:sync-token>
>> </D:multistatus>
>>
>> --localhost-29378-1383156666-1025603243
>> Date: Wed, 30 Oct 2013 18:12:47 GMT
>> Content-Type: application/xml; charset=utf-8
>> Content-Length: 375
>>
>> <?xml version="1.0" encoding="utf-8"?>
>> <D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
>>     <D:response>
>> <D:href>/dav/calendars/user/ken/Default/4E4B3490-6F01-41B9-AA5B-FE2CD6A30632.ics</D:href>
>>       <D:status>HTTP/1.1 404 Not Found</D:status>
>>     </D:response>
>> <D:sync-token>http://cyrusimap.org/ns/sync/1368011844-356</D:sync-token>
>> </D:multistatus>
>>
>> --localhost-29378-1383156666-1025603243
>> Date: Wed, 30 Oct 2013 18:14:05 GMT
>> Content-Type: application/xml; charset=utf-8
>> Content-Length: 202
>>
>> <?xml version="1.0" encoding="utf-8"?>
>> <D:multistatus xmlns:D="DAV:" xmlns:C="urn:ietf:params:xml:ns:caldav">
>> <D:sync-token>http://cyrusimap.org/ns/sync/1368011844-356</D:sync-token>
>> </D:multistatus>
>>
>> --localhost-29378-1383156666-1025603243--
>>
>> End of MIME multipart body.
>>
>> --
>> Kenneth Murchison
>> Principal Systems Software Engineer
>> Carnegie Mellon University
>>
>>
>>
>> _______________________________________________
>> tc-push-l mailing list
>> tc-push-l@lists.calconnect.org
>> http://lists.calconnect.org/mailman/listinfo/tc-push-l
>>
> _______________________________________________
> tc-push-l mailing list
> tc-push-l@lists.calconnect.org
> http://lists.calconnect.org/mailman/listinfo/tc-push-l

-- 

*Marten Gajda*
Schandauer Straße 34
01309 Dresden
Germany

tel: +49 177 4427167
email: marten@dmfs.org
twitter: twitter.com/dmfs_org

VAT Reg. No.: DE269072391