Re:URIs and digest values (was: ObjectReference shouldn't be

Interesting discussion.  The URI would essentially consist of the digest?  Would
it be 
possible to use a shortened digest as an ID/IDREF value?  For example, instead
of

<IDREF>="<random number>"

could we have

<IDREF>="<base64-encoded short digest>"

where the number of bits in the digest would determine the degree of
"randomness" in 
the ID needed to reduce the possibility of collisions?  Theoretically, the
entire digest 
could be used.  The IDREF would make it easier to locate the object.  The
referenced 
tag could still contain an indirect location.  The object would be, in a sense, 
content-addressable.

Thanks,
Rich

____________________Reply Separator____________________
Subject:    URIs and digest values (was: ObjectReference shouldn't be si
Author: Daniel LaLiberte <liberte@w3.org>
Date:       11/10/99 2:01 PM

I've accumulated a few things to say regarding identifiers, resources,
and signatures.  I hope this is useful to you.

Joseph M. Reagle Jr. writes:
 > Richard, thanks for your emails on this topic, I think it is driving
 > us to clarify a number of issues here. Thoughts follow ...
 > 
 > At 10:36 99/11/09 -0700, rhimes@nmcourt.fed.us wrote:
 >  >>How do you make an assertion sans location semantics, such that:
 >  >
 >  >>        B: some object when found and transformed yields the 
 >  >>        following DigestValue  ?
 >  >
 >  >>I think this is a valid question. We have a requirement to
 >  >>identify objects via URIs. The URI need not be a URL. (In fact, I
 >  >>think the DigestValue is a good URI). 

Technicality: A digest value by itself may not be a URI, strictly
speaking, unless it includes a prefix of the URI scheme name followed by
a ':'.  But a digest value could certainly be used in a URI.

In order to talk about some object to be used in creating a digest value
(or for any other purpose), there must be a reference to that object
using one of three kinds of methods of referring to the object.

(1) A URI (either URL or URN - no real difference in this regard, as
    argued below).

(2) Some expression for how to obtain it by some other means, probably
    grounded in URIs.

(3) The object may be available as immediate data, such as embedded in
    the same XML document as where it is being used.  

URIs are the preferred means for referring to things on the internet but
other kinds of things might be translated into URIs (such as XML element
names, or ID attribute values).  Referring to immediate data is also
relevant, especially when signing data if you can't trust the validity
of references or resolution mechanisms.

But now to the question of whether a digest value could be used in a URI
as a reference an object from which you want to compute the same digest
value.  That *could* be done, but it seems at least suspicious if not
spurious.  Seems you want an independent way of referencing the object
so that you can verify that the digest value is valid.  However, an
independent reference to the object is not necessary if it is
computationally infeasible to get the same digest value from any other
content.

On the other hand, a digest value that is used in a URI is not
particularly useful, to say the least, as a hint for resolving the URI
to get the content.  The namespace of all digest values is globally
flat, so resolution would have to be very centralized, which is
unscalable.  Hence the idea of a Location hint.  But, it might be
that a new URI scheme could combine persistent location hints and 
a digest value to get the best of both.

So we should not presume any particular characteristics of a URI
only knowing it is a URI.  It may or may not be a persistent or unique
identifier for a static or dynamic resource.   Moreover, the terms
"URL" and "URN" don't offer much clarification, even though the *intent*
of a URN is that it is persistent.

 >  >>[...] We could relax that
 >  >>requirement and remove "Location" from the signature and only
 >  >>provide it as a resolution hint;

I think it is better to just say "URI", avoiding the confusion
of whether or not it can be used as a location.

 > or 
 >  >>we permit a level of indirection pre/post signature creation. Pre:
 >  >>ObjectReference uses some mechanism such as a directory, URN,
 >  >>manifest etc. that provides resolution.
 >  >
 >  >>Post: you allow a "redirect" or "cache" statement to be associated
 >  >>with the signature that states "the URI found in ObjectReference
 >  >>resolves to X"
 >  >
 >  >I'd like to see samples of these approaches.
 >  
 > I discuss how to do it in [1] though I don't provide a syntactical
 > proposal (also see [2]). When I wrote [1] back in July, I even
 > considered dropping locations from the manifest (I'm glad this
 > conversation finally took off!)
 > 
 > However, with respect to our present design I like the requirement
 > that you have a URI NOT a URL. If you must, just include a random
 > number! Now I'm not "going charter" on you <smile> but I will say
 > that the fact the requirements document says that manifests includes
 > URIs and that there is no requirement that we provide for location
 > invariant signatures (though Solo introduced this as a design
 > principle in his draft) informs my present views on what we do. 

Some applications may need to sign and later validate the invariance of
the relationship between a URI and the content it references.  P3P as
currently specified (http://www.w3.org/TR/1999/WD-P3P-19991102) is an
example; it requires that the URI for a privacy policy be invariant with
the content of the policy.

Allowing this invariance to be signed is different from requiring that
all URIs used in signatures be invariant, however.  I don't believe such
a requirement is useful and it would overconstrain the possible uses for
XML digital signatures.  We could assume that some particular URI scheme
would maintain that invariance for you, but the very same signing and
validation would have to be done somewhere else instead.

So one way to allow the invariance to be signed is to package up the URI
with the digest of the content obtained by resolving the very same URI
(no matter how it is resolved) and sign that package.  If the URI
resolves to something else, or if the thing it resolves to is different
in any way (as determined by comparing with the digest value) then the
validation would fail.  That is what we would want for P3P.

 > I'm of a mixed mind about dropping it, but since you can sort of
 > achieve that with some random number as a URN,

Verifying the invariance of the relationship between a random number URN
and the content of the resource would involve the same process I
outlined above.  The randomness of the random number URN only makes
collisions unlikely but does not guarantee no one has abused the
relationship.  A random number URN doesn't help much with resolution
either.

 > I believe you can do what you
 > want to do using URIs and/or other XML (cache) applications but I
 > presently do not feel delivering a solution to that problem is on our
 > critical path. (I suspect that a lot of thought needs to be given to
 > the protocol semantics involved, for instance, when validating
 > SignedInfo, if the signature application enounters a 301 [3], is that
 > considered non-valid?!)

What to do with redirections and other intermediate status
(e.g. metadata of various sorts) is a good question, even when you are
merely validating a signature involving resolution of any URIs.

 >  >We need the signature to remain valid during a process where the
 >  >document undergoes location changes.
 > 
 > Don't use a URL if you don't want to. 

Or equivalently, use a URL in a way that ignores the location hints
contained in it, which is perfectly valid.    Just because a URI says
"http:" doesn't mean you (a client) have to use HTTP to resolve it.

Somehow you need to use a reference associated with a document to get
the document, no matter what you call that reference or how it is
resolved.

 > At the FTF we've agreed to
 > change "Location" to "Target" at least, and maybe even distinguish
 > and use "URI" or "IDREF" as appropriate so as to stop confusing
 > people as to whether it is a URL or URI.  .

Whatever it takes to stop being confused about URL vs URN vs URI...

-----

Validating that you've got the right content when the content can change
is a more difficult problem.  Has it been considered?  In that case, you
have to validate that you have the right service that is giving you the
content.  It is probably not enough to let the service do the validation
of the signature for you unless you trust you are talking to the right
service, in which case you might as well trust that you will be given
the right content.  Or maybe you trust that it is the right service but
sometimes it screws up.  Same thing: you can't really trust it.  You
also may not trust something between you and the service.

-----

If people have issues regarding URIs that are not so closely related
to digital signatures, I suggest such discussion continue on the
uri@w3.org list, or just email me.

-- 
Daniel LaLiberte
liberte@w3.org