RE: Proposal: WebDAV and transactions - stepping back to requirem ents

I agree with Dennis that everyone should agree on what transactions actually
are. To contribute to that discussion, I think we can all agree that
transactions satisfy the well know ACID properties:

A - atomic
C - concurrent
I - independent
D - durable

Atomic: 
Either the entire transaction is reflected in the repository or none of it
is, not matter what happens (including software and hardware successes or
failures). There is no action on the part of the client to achieve
atomicity. The server must guarantee atomicity when executing a transaction.
Thus, if the repository is partially updated and there is a system crash or
an error condition arises in the conditional processing, then the
transaction is rolled back (i.e., undone, aborted). That includes both its
updates and its transaction level locks.

Concurrent: 
Multiple transactions against the repository can run concurrently.

Independent: 
This is absolutely critical: Transaction independence means that any
transaction that is programmed so that when it is running alone against the
repository it is correct, will remain correct when it is run in an arbitrary
concurrent mix of transactions against the repository. That requires two
phase locking at the transaction level. In the first phase, locks are
accumulated. In the second phase (which is the duration of either the
end-transaction method or the abort-transaction method) all of the locks are
released. The transaction level locks guarantee that the transaction can
make correct decisions based on the contents of the repository. The
transaction level of locking is at a lower level than the WebDAV LOCK and
UNLOCK methods. Those methods are like RCS locks in a source code control
system -- the WebDAV locks can persist for days, weeks, or months.
Transaction level locks on resources in the repository persist only for the
duration of the transaction. (To further the RCS analogy, the RCS checkout
and checkin operations are transactions.) Because locking is required,
deadlocks are possible. Therefore, transaction "code" must process deadlock
errors. (Deadlock errors should be retried by restarting the transaction
from the beginning. The server guarantees that retrying deadlock errors will
not result in an infinite waste of processing -- progress will always be
made in the overall mix of transactions.)

Durable: 
Once the client receives notification that the transaction has completed
successfully, the transaction is reflected permanently in the repository,
even if there is a server crash immediately afterward. In general that
requires the server to log the changes the transaction makes, and, when
restarting after the crash, to apply the redo and undo log records (i.e.,
the after and before images) to the repository. The recovery process makes
the repository reflect all the transactions that were completely finished,
but reflect nothing at all from the transactions that were in progress when
the crash occurred. Two phase locking is necessary for application of the
recovery log to get the repository back into the exact state it was when the
system crashed as opposed to some other state.

Alan Babich

-----Original Message-----
From: Dennis E. Hamilton [mailto:dennis.hamilton@acm.org] 
Sent: Friday, September 13, 2002 8:31 AM
To: w3c-dist-auth@w3.org
Cc: Babich, Alan; Roy T. Fielding; Jim Whitehead; Gary Cowan; 'B. Shadgar';
Pill, Juergen
Subject: Proposal: WebDAV and transactions - stepping back to requirements

1.	WHAT ARE THE REQUIREMENTS?

It seems to me that the conversation on transactions is looking at different
approaches to something that might be transactions and comparing those
possible implementations without being sure that we are talking about the
same concept for transactions and whatever the requirements might be in the
context of WebDAV.

I think it is very valuable at this point to stand back and investigate
exactly what is required for transactions and what the constraints on their
support would be for WebDAV.

Here are some things that I noticed in thinking about this.  Subject-matter
experts would have something more useful to say:

2.	WHAT ARE TRANSACTIONS?

I think there needs to be agreement at this level.  This is a key point of
understanding before being able to proceed.

Here is what occurs to me:

2.1	Transaction protection is a way to assure that a persistent store
(database, etc... ) never presents an unrecognized inconsistent state, and
the failure modes always revert to a consistent state.  My understanding is
that in the general case the notion of consistency is beyond the consistency
of the store abstraction.  It is there to support the consistency required
by the application in which the store is used instrumentally.  The
consistency at the granularity of the store abstractions (e.g., WebDAV with
locking) is assumed to be handled already.  Now we are talking about
consistency in coordinated use of a store for satisfying requirements that
are operable at a different level of abstraction.

2.2	Put simply and quite generally, a transaction looks atomic to
everyone
but the originating process, and it looks atomic to the originating process
outside of the critical section of the transactioned process.  And the
effect is to achieve a consistent state to all observers outside of critical
sections (the one place where rules are broken in order to be implemented).
And it is requirements of the implemented application that govern what those
consistency/atomicity conditions are and what their satisfaction is.

2.3	Transaction Processing models are highly abstracted, because we are
constrained from knowing all the particular ways that transactions may be
required in securing the consistent operation of an application.  So support
is utilitarian.  There will be limitations on the scope of support, and
understanding what those are and what the cost of going beyond it (as well
as the economy of staying within it) needs to be understood.

3. 	REQUIREMENTS FOR TRANSACTION-PROCESSING SUPPORT

Key things that I think have to be agreed around requirements for
transactions are.   I am not that great in thinking at the requirements
level, so if these don't read like requirements bear with me.  Consider
instead what might be the requirement behind quasi-requirements like these:

3.1.	What is the required atomic appearance of consistency outside of
critical (transaction-protected processing) sections?  (Express as a
principle and what people are to get out of it, not so much as a given
solution.)

3.2.	What is the required appearance of consistency within critical
sections?  (E.g., do changes to an underlying store appear to have taken
place just as they would if transaction-protection were not requested?)

3.3.	What is the ability to integrate in a transaction regime that
extends
beyond the operation of a single WebDAV service?  Related to scope of
support (3.5) and

3.4.	The unobtrusiveness of transaction support in the expression of
transaction-protected processes (e.g., that there be little or no difference
beyond what it takes to signify that an operation is protected under an
identified transaction) - only expressed at the level that doesn't foretell
an implementation or protocol approach

3.5.  What is the scope of required consistency support (the place where
use-cases might help the most).

4.	CONSTRAINTS

There are constraints that will apply to the acceptable introduction of
transaction-processing support in the WebDAV protocol.  These limit the
solution space available for responding to requirements.  (Again, I am not
that practiced at differentiating between constraints and requirements.  I
recognize value in doing so.)  Here are some candidates:

4.1.	Actually using an (existing) transaction model at all.  I notice in
(3)
I am presuming that transaction-protection techniques are required.  I do
think that is a given, because it is an integrative requirement (integrating
with available rather-standard mechanisms, integrating from the standpoint
of the knowledge and experience required to develop and validate
applications).  But the requirement for that or the constraint of that needs
to be surfaced.

4.1.	Supporting provisions must honor the STEP model and the spirit of
HTTP
to no lesser degree than WebDAV already does (that is, simply tunneling
another protocol wouldn't count)

4.2.	If there is a way to provide a scaled-down, high-value case, it must
be
convincing and not merely intended that it does scale.  (I don't know how
you do that without demonstrating a full solution, but who knows ... .)

4.3.	And scaling between non-support and (degrees of) support.  Like,
reusing already-defined methods and supplementing their request bodies or
something like that, wherever possible.  Something about failure cases in
this area, along with conditions on use of HTTP and WebDAV error status,
etc.

-- Dennis

-----Original Message-----
From: w3c-dist-auth-request@w3.org
[mailto:w3c-dist-auth-request@w3.org]On Behalf Of Jim Whitehead
Sent: Thursday, September 12, 2002 12:58
To: Gary Cowan; w3c-dist-auth@w3.org
Subject: RE: Proposal: WebDAV and transactions

> Well I suppose, but isn't this what wrapping updates with a
> Lock/Unlock pair is intended to accomplish. I have nothing
> against transactions, I just don't have any need for them
> spanning multiple WebDAV methods. Yet:)

Lock/unlock doesn't give you rollback capability. Even though you're the
only person who can write to a resource during a lock, it still may be the
case that you're unable to complete the PROPPATCH part of a PUT/PROPPATCH
pair. The TCP connection may go down, the server may go down, you may be
exceeding your quota, etc.

Now, if the total number of use cases is relatively small, it probably makes
more sense to create new methods explicitly for these use cases, instead of
a general purpose transaction mechanism.

- Jim


-----Original Message-----
From: w3c-dist-auth-request@w3.org
[mailto:w3c-dist-auth-request@w3.org]On Behalf Of Roy T. Fielding
Sent: Thursday, September 12, 2002 07:01
To: Babich, Alan
Cc: w3c-dist-auth@w3.org
Subject: Re: Proposal: WebDAV and transactions

[ ... ]

That's not how I described transactions.  I said that the requests are
collected on the server and only applied when the commit message is
received.  If the state has changed such that the methods are no longer
applicable, then the method transaction will fail on commit.

....Roy

-----Original Message-----
From: w3c-dist-auth-request@w3.org
[mailto:w3c-dist-auth-request@w3.org]On Behalf Of Babich, Alan
Sent: Monday, September 02, 2002 13:59
To: 'B. Shadgar'; Pill, Juergen; w3c-dist-auth@w3.org
Subject: RE: Proposal: WebDAV and transactions

One thing you do NOT want under any circumstances are
begin-transaction, end-transaction, and abort-transaction
methods.

[ ... ]

Received on Friday, 13 September 2002 15:55:01 UTC