Etag-on-write, draft -04

   Julien, Lisa, and I have been having a discussion about this  
offline, which should be recorded here.  Here is a glommed-together  
version of the discussion so far.

	-wsv


On Nov 28, 2006, at 4:09 PM, Lisa Dusseault wrote:

> Wilfredo, have you thought about the backwards-compatibility issue  
> with introducing a new header in order to ensure the same behavior  
> that's common now and that existing clients depend on?  I think that  
> issue might hit your client the worst, as well as impacting Xythos  
> WFC, but I don't know exactly how the Mac client works.
>
> I mentioned this on the HTTP list Sept 6 and 12, and Jamie Lokier  
> wrote a far better explanation of the tradeoffs on Sept 12,  
> explaining how the tradeoffs depend on how deployed clients behave  
> as well as deployed servers.   Roy chimed in suggesting a solution  
> that manages the case Jamie outlined where we try to be compatible  
> with deployed clients.  Jamie replied (Sept 14) with a slight  
> variation on that solution that shows how easy it would be for  
> servers to implement.  I believe Roy's solution to be the best I've  
> seen though it requires a little elaboration about what "can use in  
> in future conditional requests" means.
>
> http://lists.w3.org/Archives/Public/ietf-http-wg/2006JulSep/

On Nov 28, 2006, at 4:27 PM, Julian Reschke wrote:


> The Xythos client always assumes that content isn't rewritten. And  
> if no ETag is returned, it uses the Last-Modified date as cache key.  
> So it's already broken with respect to servers that have to rewrite.
>
> Roy never made a proposal, and didn't answer when asked for  
> clarification/confirmation.
>
> Assuming he meant that the server is supposed to make sure that  
> strong etags work as (IMHO broken) clients expect: that wouldn't  
> cover the case where the server *has* to rewrite content, such as  
> when the backing store doesn't provide the capability of storing  
> binary content (think XML or ICS). I think we all agree that in  
> cases like these we still want the ability to do an PUT/If-Match  
> request, and that's what the extension header in my draft provides  
> (the client gets the etag upon PUT but with a warning attached, and  
> then it's the client who can decide whether it wants to refetch the  
> content or not).

On Nov 28, 2006, at 5:10 PM, Lisa Dusseault wrote:


> Yes, we've been through this before -- a server that has to rewrite  
> content, so it issues a "not-valid-etag" in the ETag header in the  
> response to the PUT request, then issues a *second* ETag for the  
> rewritten content (this could go in a separate new header for  
> optimization with future clients) would work perfectly with Xythos  
> WFC.
>
> In other words, even for a client that assumes that servers do not  
> rewrite, like Xythos WFC, there's an easily-implementable way for  
> servers to that do rewrite to have backwards compatibility.
>
> [Re: Roy's proposal]
>
> Well, Jamie elaborated on what I describe as Roy's proposal.  I may  
> have been reading more into Roy's email than you did, but apparently  
> Jamie understood it the same way I did.
>
> But your proposal only works with upgraded clients.  Roy's proposal,  
> or Jamie's or my understanding of it, works with existing deployed  
> clients as well because it mimics the behavior of a change-by-a- 
> client even though it's a change-by-a-server-after-PUT.

On Nov 29, 2006, at 1:10 AM, Julian Reschke wrote:


> I'm not sure I follow.
>
> So are you proposing to return a dummy etag (that will always fail  
> If-Match tests) upon PUT, in order to fix clients that make an  
> assumption not licensed by RFC2616? It seems to me that it would be  
> easier to fix the client (which is in active development), instead  
> of having to change potentially many servers.
>
> [Re: easily-implementable wayto have backwards compatibility]
>
> Well, if you really think this is a good idea, please go ahead and  
> write down a specific proposal, like I did :-).
>
> [Re: Roy/Jamie proposal]
>
> 1) I think we do agree that there is an interoperability problem  
> that needs to be solved. There are clients out there not doing the  
> right thing, because the server behaves in a way they don't expect.
>
> 2) As far as I can tell, the behavior of these servers is legal  
> according to RFC2616. This is explained in <http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-04.html#rfc.section.1.3 
> > -- in the last months I haven't seen not a single comment stating  
> that this analysis is incorrect. If you think it's incorrect, please  
> be more specific.
>
> 3) We can't change deployed code, and we can't make incompatible  
> changes to RFC2616.
>
> Thus, I think the best thing we can do is to clarify the situation,  
> and add code that make things easier for future clients. And that's  
> exactly what my document does.

On Nov 29, 2006, at 10:58 AM, Wilfredo Sánchez Vega wrote:


>  [Re: original question]
>
>   When that behavior is known the be broken, I guess I don't really  
> sympathize with the clients in question.  A DAV server is not  
> required to store content octet-by-octet.  Nothing says that they  
> should, and nothing the server does says that it did.
>
>   By "your client", I assume that you mean WebDAV FS.  (We have  
> others.)  WebDAV FS does in fact depend on octet-by-octet storage,  
> as far as I can tell (I don't work on it, per se).  But WebDAV FS  
> expect WebDAV to behave like a file system.  A WebDAV server that  
> does not behave like a file system will not work with WebDAV FS.  I  
> do not take this to mean that all servers should behave like a file  
> system, but rather that they should if that's how they want to be  
> used.
>
>   A server that does content modification on PUT will make WebDAV FS  
> unhappy no matter what the headers say.  So don't use WebDAV FS as a  
> client on those servers.  It's not appropriate.  But right now,  
> WebDAV FS has no way to distinguish between a server that does  
> content modification and one that does not.  This header allows a  
> client to do so, and you can then expect that WebDAV FS should, if  
> the authors care to, use that information to do something more  
> useful when it detects content modification.  Perhaps it can do a  
> reload.  Perhaps it can tell the user that it's using a server that  
> isn't supported.  I don't know, but I don't really care that much,  
> because the servers that behave like a file system will continue to  
> do so and will continue to work just fine.
>
>   Let's not assume that those are the only kinds of servers that we  
> need in the world, and accept that not all DAV clients are  
> appropriate for all DAV servers.  It's already the case today, and I  
> don't see this changing, or even as a problem.
>
>   CalDAV clients aren't going to find vanilla DAV servers all that  
> exciting.  And I certainly don't expect that the iCal server I'm  
> writing needs to support authoring via WebDAV FS in a calendar  
> collection.  It's a ridiculous use case.
>
>   ETag behavior on WebDAV FS is already broken in the most common  
> case, by the way.  I frequently have a document open on a DAV  
> volume, and a second after I save it, my editor thinks the document  
> changed (because Apache changed the ETag that it never should have  
> sent on PUT in the first place).  Thus, the most common case (Apache  
> server) is already broken today, partly for reasons (the weak- 
> >strong change) that are not addressed in this draft at all, because  
> they are unrelated, and arguably an Apache implementation problem.
>
>   My point being that as a server implementor, I have some clients  
> doing one thing, and other clients doing another.  Changing behavior  
> annoys some and make others happy.  Meh.  Give me a standard so that  
> at least new clients can behave themselves sanely.

On Nov 29, 2006, at 1:38 PM, Lisa Dusseault wrote:


>  [Re: no sympathy for broken clients]
>
> That's arguably true (OTOH arguably an unstated assumption that was  
> reasonable for those clients to make), but maybe we can do something  
> sensible with those clients anyway rather than argue what was meant  
> by an old spec and assumed by its authors and readers.
>
>
>>  A server that does content modification on PUT will make WebDAV FS  
>> unhappy no matter what the headers say.
>
> I'm just not understanding everything, putting this statement  
> together with the rest of your email.  In this statement, you imply  
> that even if that server treats content modification on or after a  
> PUT exactly the same as it would a content modification by a client  
> after the PUT -- by issuing one ETag in the response to the PUT, and  
> another ETag immediately afterward.
>
> Then later on in your email you explain how Apache issuing first a  
> weak ETag, then a strong one, makes WebDAV FS think that the  
> document has changed.  This sounds perfect for what has been  
> suggested -- that if a content-modifying server issued a strong ETag  
> and then another strong ETag, WebDAV FS would think the document has  
> been changed, which it has.
>
> I don't understand why there's a problem with this approach.  It  
> works well with deployed clients.  It's perfectly compatible with  
> complementary approaches that also advertise that the server does  
> content modification and that modification has been done in this case.

On Nov 29, 2006, at 1:58 PM, Julian Reschke wrote:


>> That's arguably true (OTOH arguably an unstated assumption that was  
>> reasonable for those clients to make),
>
> For the record: I do not think it was a reasonable assumption.
>
>> but maybe we can do something sensible with those clients anyway  
>> rather than argue what was meant by an old spec and assumed by its  
>> authors and readers.
>
> Hm, so you *do* want to make an incompatible change to RFC2616?
>
>>>   A server that does content modification on PUT will make WebDAV  
>>> FS unhappy no matter what the headers say.
>>
>> I'm just not understanding everything, putting this statement  
>> together with the rest of your email.  In this statement, you imply  
>> that even if that server treats content modification on or after a  
>> PUT exactly the same as it would a content modification by a client  
>> after the PUT -- by issuing one ETag in the response to the PUT,  
>> and another ETag immediately afterward.
>
> ...I think there's something missing in this sentence. If it does  
> this that, what then?
>
>> Then later on in your email you explain how Apache issuing first a  
>> weak ETag, then a strong one, makes WebDAV FS think that the  
>> document has changed.  This sounds perfect for what has been  
>> suggested -- that if a content-modifying server issued a strong  
>> ETag and then another strong ETag, WebDAV FS would think the  
>> document has been changed, which it has.
>
> But that behaviour breaks all clients that indeed would work well  
> with "harmless" content rewriting, such as when ICS entities are  
> normalized. Those clients could go on editing the resource, because  
> the transformation by the server obeys the rule:
>
> 	t(e2(e1(c))) = t(e2(t(e1(c))))
>
> (<http://greenbytes.de/tech/webdav/draft-reschke-http-etag-on-write-04.html#rfc.section.1.2.p.11 
> >)
>
>> I don't understand why there's a problem with this approach.  It  
>> works well with deployed clients.  It's perfectly compatible with  
>> complementary approaches that also advertise that the server does  
>> content modification and that modification has been done in this  
>> case.
>
> It breaks existing clients that do not care about servers rewriting  
> the content the same way every time the resource is PUT. This is  
> *only* a problem when the client tries to do a GET/range request. I  
> think neither Xythos WFC and Apples' WebDAV client fall into that  
> category, though.

On Nov 29, 2006, at 3:13 PM, Wilfredo Sánchez Vega wrote:


>> That's arguably true (OTOH arguably an unstated assumption that was  
>> reasonable for those clients to make),
>
>   I will strongly disagree that this was a reasonable assumption to  
> make.  It is well known that a DAV server is allowed to rewrite  
> content, and nothing in any spec that I'm aware of says otherwise.
>
>>>   A server that does content modification on PUT will make WebDAV  
>>> FS unhappy no matter what the headers say.
>>
>> I'm just not understanding everything, putting this statement  
>> together with the rest of your email.  In this statement, you imply  
>> that even if that server treats content modification on or after a  
>> PUT exactly the same as it would a content modification by a client  
>> after the PUT -- by issuing one ETag in the response to the PUT,  
>> and another ETag immediately afterward.
>
>   What is the point of issuing an ETag on PUT that is not useful for  
> any request that follows?  It's completely misleading because it  
> implies that the client has something that should be cached, when it  
> fact it already has a stale copy.  That's bogus.
>
>> Then later on in your email you explain how Apache issuing first a  
>> weak ETag, then a strong one, makes WebDAV FS think that the  
>> document has changed.  This sounds perfect for what has been  
>> suggested -- that if a content-modifying server issued a strong  
>> ETag and then another strong ETag, WebDAV FS would think the  
>> document has been changed, which it has.
>
>   WebDAV FS isn't the only DAV client in the world, and it's not  
> even a very interesting one.  The correct thing for a server to do  
> with today's WebDAV FS is not return an ETag on PUT, and let it use  
> the last-modified header instead.
>
>   This has the pleasant effect of not breaking clients that don't  
> make the same bad assumptions that WebDAV FS is making.
>
>> I don't understand why there's a problem with this approach.  It  
>> works well with deployed clients.  It's perfectly compatible with  
>> complementary approaches that also advertise that the server does  
>> content modification and that modification has been done in this  
>> case.
>
>   My point with the Apache weak->strong problem is that no, the  
> status quo does not work well today with the most popular web server  
> in use today and with what is probably one of the most widely used  
> DAV authoring clients in use today, and we should stop pretending  
> that it does.

On Nov 29, 2006, at 3:45 PM, Lisa Dusseault wrote:

>>   What is the point of issuing an ETag on PUT that is not useful  
>> for any request that follows?  It's completely misleading because  
>> it implies that the client has something that should be cached,  
>> when it fact it already has a stale copy.  That's bogus.
>
> So what would be the client behavior if the PUT response contained  
> no ETag in the ETag header (and no last-modified header)?  Then the  
> client could not assume that the content had not been modified,  
> which would be correct behavior.  Later if the client does a HEAD,  
> PROPFIND or other, the server could provide the post-modification  
> ETag.
>
> This could be less bogus than the proposal for a "never-valid" ETag  
> that's not useful for any request that follows.
>
> What I'd like to come up with is proposals that allow a new client  
> to work well with existing servers, and a new server to work well  
> with existing clients.  Sometimes there are cheap things you can do  
> to make those cases work even if there have been bad assumptions in  
> the past.
>
> Right now we have three interesting things to flag (no change,  
> harmless change, big change) and our existing tools only allow us to  
> flag two of those things.  The major question is whether "harmless  
> modifications" should be flagged in such a way so that:
> 	1. Deployed clients can't distinguish a harmless modification from  
> the case where no modification occurred; only upgraded clients can  
> tell the difference by using a new header
> 	or 2.  Deployed clients can't distinguish between a harmless  
> modification and a major modification done perhaps by another  
> client; only upgraded clients can tell the difference by using a new  
> header
>
> I believe the proposal in the draft describes case 1, whereas case 2  
> would be safer.
>
> In any case, major server modifications (non-harmless) should never  
> treated in such a way that they appear as if no change occurred to  
> deployed-base clients.  Do we generally agree on that?