RE: collection with ordered members

As I see it, you raise two objections to ordered collections. (If you have
others, I missed them, for which I beg your pardon.)  Let me try to refute
them as best I can.  First an outline, then details.  

1. "The promise to maintain order is a serious burden to implementers."  
2. "The feature ... breaks older clients".

I also have the impression that you see a third objection but I can't find
a quote that states it well.  (I use quotes to ensure I am not attacking a
straw-person.)  I think your objection might be

3. It has never been done before.

So before I proceed, are these your three objections, or did I
misunderstand them?  Are there others?

Either of the first two would be severe.  I hope to show that neither one
applies.  I hope my formal tone won't be taken as harsh, I am simply trying
t be as logical as I can, and to ensure I address every argument you make.

Re 1 (expensive)

A. It is not a serious burden to implementers

You object that "Most DAV servers will most likely be implemented on top of
file systems", and, as I concede, file servers don't allow you to control
the order by which a directories components are retrieved.  Fair enough.  But

1. DAV requires several other features that file systems don't support.  

  It requires properties and it requires external members.  I know of no
file system that allows one to store arbitrary properties with a file
system object.  Maybe the Macintosh does this, but who uses that?  Unix
does not, and, while the CIFS spec mentions extended attributes, I can't
see any trace of this in NT4.  Nor is it clear what one writes into the
file system to store an external member.

2. component order can be implemented straightforwardly and cheaply.

One method is to write a file, something like "order.dat", that records the
order.  Indeed, this can be a cache of the data that would be returned by
an INDEX on the collection.  Those operations that would add new elements
would have to edit this file, but that's not hard.

I can't tell from your message, but you might have the impression that I
(we) are asking for the ability to request an *arbitrary* sorting of the
members at run time. If this were so, that would be expensive, since no
cache would work.  You write: 

>... the server has to read in the entire directory, keeping it in
>local memory, sort according to the requested order, and then return it.

But that is not what I am asking for.  What I am asking for is ONE order.

3. Some DAV servers will be implemented on top of document management
systems that do support ordering directly.  Furthermore, I would claim that
those who want really large collections are MORE likely to use such
implementations.  (By analogy, it's said that Apache is the most widely
used Web server.  But I am not sure that it is as widely used by those with
truly huge Web sites.)  If DAV does not provide means to control order,
then it make DAV much less interesting to those who need the ability.  They
will seek other protocols.  

1 B. Even if it were expensive, it could be made optional.

I propose that an implementation be free to return members in any order it
likes, and free to reject requests with headers that refer to ordering.
From my standpoint, this won't hurt a bit.  If the Brand X server does not
support it, I'll just use one that does.

You have objected before that optionality is bad, but we have plenty of
other optional areas in DAV, at least according to the current available
draft.  PATCH supports VTML, optionally.  Support for locking is optional
(level 2), and servers are allowed to support "any combination of exclusive
and shared locks".  Versioning is optional (level 3) .  In this case, I
don't even care whether there's a specified means to detect whether a
server supports ordered collections.  I'll settle for getting an error.

Re 2 (incompatible)

You write

>the feature ... breaks older clients. An older client may fully
>understand the compound document format being used but because it was
>written before DAV it would not be able to put the pieces back together.
>Whereas if it could simply do a GET on the resource and get back the entire
>document, then both DAV and non-DAV HTTP aware clients can interact with
>compound documents. Of course, that requires the server to be aware of the
>particular compound document format in use which reduces the situation to
>the case I mentioned before, which requires discovery for the particular
>formats the server supports, thus arguing for putting this feature in as an
>extension and not as part of the base DAV spec.

This is, I think, an elaboration on the idea that a client would write a
compound document to the server with a PUT (perhaps as a MIME multipart, or
perhaps not), that the server would understand the document format,
disassemble it into pieces, store it in a collection, then serve up the
pieces (piecemeal) via GETs.

If that's the idea, I think it's a strawperson position, or at least, it's
not what *I* want.  If someone else does, let them speak up.  I have not
proposed this, and I have no interest in it.  In my view, a client can
either PUT and GET a compound document in a format that it  (privately)
understands, or do a MKCOL and a set of ordered PUTs, but not both.

Re 3: (never done before)

One can find support for ordered collections in many large scale document
repositories (e.g. Documentum), in the DMA API, and in the POP and IMAP
mail APIs.  

It is also my impression that DAV has some features never seen before, as
well.  Are State Tokens not an innovation?  (and a clever one, I might add)
 And Lock Tokens?

In summary, ordered collections can be implemented as cheaply as other key
features in DAV, offer no significant performance degradation, are entirely
optional, fully compatible with all existing applications, are already well
established (in other protocols).  They would make DAV significantly more
useful for certain applications, at only a very small cost in complexity.

If you find my reasoning unsound or based on falsehood, I would be pleased
and in your debt to be corrected. If your objections arose from a
misunderstanding of the idea, I apologize for proposing it so poorly
before, and if I now have your support, I could quickly generate a formal
description.

with all best regards,

Jim

Received on Tuesday, 28 October 1997 21:31:24 UTC