Re: ETags, If-Match and database versions

On Sat, Dec 28, 2024 at 12:08:02PM +0000, Graham Cox wrote:
> >
> On Sat, 28 Dec 2024 at 00:51, Roy T. Fielding <fielding@gbiv.com> wrote:
> > The good news is that unless you deliberately coded the
> > representation-generating algorithm to be random, it is probably already
> > consistent over time and would have no problem using strong etags. The
> > backend doesn't have an opinion -- only the HTTP interface matters.
> 
> 
> So, this is the bit that really sparked things for me here. We've always
> done exactly this - assumed that the same input data would always produce
> the same JSON, and therefore could just use the database version as a
> strong etag. And, it turned out, we were proven wrong. We did a *minor*
> version update of our underlying framework (Spring Boot 3.3 to 3.4), which
> it seems had some knock-on effect down the line that changed the order in
> which the fields were rendered into the JSON object - which would therefore
> require a different strong etag for the exact same input data.
> 
> I've also seen the suggestion to append a server version to the etag to
> avoid this, but there are significantly more reasons to change the server
> version that *wouldn't* change the JSON representation. And doing that
> would then mean every time the server version changes then all the etags
> would change, which would mean that caching and optimistic locking breaks -
> often needlessly...
> 
> > As the origin server developer, you define the consistency of your
> > resources. It doesn't matter how they are stored. It matters how
> > consistently you represent them across the interface. If you are consistent
> > for a given resource, that resource can use If-Match with strong etags. If
> > you are inconsistent, then those features of HTTP have to be disabled.
> > Hence, If-Match is not allowed with weak etags because no match can be
> > guaranteed.
> 
> 
> This is the crux of it. And maybe I'm being over-cautious with it all -
> which wouldn't exactly be unusual for me! 🙂 - but following that through,
> it would seem that if I *can't* 100% guarantee that the same resource will
> always be rendered byte-for-byte into the same response body - e.g. because
> an underlying framework upgrade has changed something that I didn't realise
> - then I should be using weak etags.

If you wish to have someone else do the thinking and make the guarantees
(100% ? lol), then you should not be making decisions about ETags.

If generating a proper Strong ETag is critically important, then you
should err on the side of caution and change ETag whenever any part of
your system changes, and acknowledge that caching efficiency may be
impacted.

If you instead have some level of *TESTING* and can have a reasonable
degree of confidence for your test cases that the responses prior to
application of a set of changes are identical to the responses after
application of the change set, then you can use Strong ETags.  You
might add a version component to the ETag which you *manually* increment
when your test suite determines that the responses have changed after
the application of a change set.

> And I fully understand that certain features can't reliably be used with
> weak etags - such as byte-range requests. I'm just not sure why If-Match
> preconditions, and therefore optimistic locking, should be on that list. If
> anything, it seems to me that this is potentially a *better* candidate for
> allowing weak etags than caching is - the backend server can safely perform
> the update as long as the original data hasn't changed, regardless of the
> byte-for-byte representation that was used.

If-Match (with proper Strong ETags) is important for ensuring that an
operation requested by the client on a resource is applied to the exact
same resource on the server.  e.g. DELETE a specific resource only if
that resource has not been changed by someone else; or publish exactly
this resource to production, and not any other one; COPY, MOVE, etc.
Give the client HTTP status 412 Precondition Failed if the If-Match
condition fails.

Cheers, Glenn

Received on Saturday, 28 December 2024 13:17:28 UTC