INTERNET-DRAFT Saveen Reddy, Microsoft draft-reddy-dasl-protocol-01.txt Del Jensen, Novell Surendra Reddy, Oracle Rick Henderson, Netscape Jim Davis, Xerox Alan Babich, Filenet Expires Oct 3, 1998 April 3, 1998 DAV Searching & Locating Status of this Memo This document is an Internet draft. Internet drafts are working documents of the Internet Engineering Task Force (IETF), its areas and its working groups. Note that other groups may also distribute working information as Internet drafts. Internet Drafts are draft documents valid for a maximum of six months and can be updated, replaced or obsoleted by other documents at any time. It is inappropriate to use Internet drafts as reference material or to cite them as other than as "work in progress". To learn the current status of any Internet draft please check the "lid- abstracts.txt" listing contained in the Internet drafts shadow directories on ftp.is.co.za (Africa), nic.nordu.net (Europe), munnari.oz.au (Pacific Rim), ftp.isi.edu (US East coast) or ftp.isi.edu (US West coast). Further information about the IETF can be found at URL: http://www.ietf.org/ Distribution of this document is unlimited. Please send comments to the Distributed Authoring and Versioning (WEBDAV) working group at , which may be joined by sending a message with subject "subscribe" to . Discussions of the WEBDAV working group are archived at . Abstract This document specifies a set of methods, headers, and content-types composing DASL, an application of the HTTP/1.1 protocol to efficiently search for DAV resources based upon a set of client-supplied criteria. Fielding, et al [Page 1] INTERNET-DRAFT DASL Friday, March 13, 1998 Table of Contents DAV SEARCHING & LOCATING..................................1 1.INTRODUCTION...........................................4 1.1. Relationship to DAV.................................4 1.2. Terms...............................................4 1.3. Notational Conventions..............................4 1.4. DASL in Operation...................................4 2.THE SEARCH METHOD......................................4 2.1. Overview............................................4 2.2. Query Transport vs. Query Semantics and Query Grammar5 2.3. The Request.........................................5 2.3.1.The Request-URI as Search Arbiter.................5 2.3.2.The Request Body..................................5 2.4. The Response........................................5 2.5. Response Extensions.................................5 2.6. Example.............................................6 2.7. The DAV:searchrequest XML Element...................6 3.DISCOVERY MECHANISMS...................................7 3.1. Query Grammars......................................8 3.2. Query Grammar Schema Discovery......................8 4.THE OPTIONS METHOD.....................................9 4.1. Overview............................................9 4.2. Request.............................................9 4.3. Response............................................9 4.4. Example.............................................9 5.NEW HEADERS...........................................10 5.1. The DASL Response Header...........................10 5.1.1.Overview.........................................10 5.1.2.Syntax...........................................10 5.1.3.Example..........................................10 6.THE DAV:SIMPLESEARCH GRAMMAR..........................10 6.1. Introduction.......................................10 6.2. Example............................................10 6.3. DAV:select.........................................11 6.4. DAV:from...........................................11 6.5. DAV:where..........................................11 6.5.1.Example..........................................12 6.5.2.Example..........................................12 6.6. DAV:sortby.........................................12 6.7. Properties.........................................12 Reddy, et al [Page 2] INTERNET-DRAFT DASL Friday, March 13, 1998 6.7.1.DAV:rank.........................................12 6.8. Search Operators...................................13 6.8.1.Introduction.....................................13 6.8.2.DAV:and, DAV:or, DAV:not.........................13 6.8.3.DAV:eq...........................................13 6.8.4.DAV:lt, DAV:lte, DAV:gt, DAV:gte.................13 6.8.5.DAV:contains.....................................13 6.8.6.DAV:literal......................................14 6.9. Typing.............................................14 6.10. Variants..........................................15 6.11. Query Schema for DAV:simplesearch.................15 6.12. Versioning........................................15 6.13. Internationalized Content.........................15 7.SECURITY CONSIDERATIONS...............................15 8.AUTHENTICATION........................................15 9.IANA CONSIDERATIONS...................................15 10. CHANGE HISTORY.......................................15 11. REFERENCES...........................................16 12. AUTHOR'S ADDRESSES...................................16 Reddy, et al [Page 3] INTERNET-DRAFT DASL Friday, March 13, 1998 1. Introduction This document describe a set of methods, headers and content-types forming DASL, an application of HTTP/1.1 that allows clients to perform searching operations on the properties and content of DAV resources. DASL is a lightweight search protocol to transport queries and result sets and allows clients to make use of server-side search facilities, the motivation for which is described by [DASLREQ]. DASL includes the SEARCH method, the DASL response header for use with the OPTIONS method, the DAV:searchrequest XML entity, and the DAV:simplesearch query grammar. 1.1. Relationship to DAV DASL relies on the resource and property model defined by [WEBDAV]. DASL does not alter this model. Instead, DASL allows clients to access DAV- modeled resources through server-side search. 1.2. Terms This draft uses the terms defined in [RFC2068], [WEBDAV], and [DASLREQ]. 1.3. Notational Conventions The augmented BNF used by this document to describe protocol elements is exactly the same as the one described in Section 2.1 of [RFC2068]. Because this augmented BNF uses the basic production rules provided in Section 2.2 of [RFC2068], those rules apply to this document as well. The key words "MUST", "MUST NOT", "REQUIRED", "SHALL", "SHALL NOT", "SHOULD", "SHOULD NOT", "RECOMMENDED", "MAY", and "OPTIONAL" in this document are to be interpreted as described in [RFC2119]. 1.4. DASL in Operation As an application of HTTP/1.1, DASL follows the HTTP/1.1 request/response chain. A client invokes the SEARCH method and includes a text/xml entity containing a query. The server responds with a text/xml entity that contains the corresponding result set. It is an objective of DASL to minimize the complexity of clients so as to facilitate widespread deployment of applications capable of utilizing the DASL search mechanisms. 2. The SEARCH method 2.1. Overview The client invokes the SEARCH method to initiate a server-side search. The text/xml body of the request defines the query. A successful response is a text/xml entity matching the [WEBDAV] PROPFIND response. Reddy, et al [Page 4] INTERNET-DRAFT DASL Friday, March 13, 1998 2.2. Query Transport vs. Query Semantics and Query Grammar The SEARCH method provides a framework for the transmission of a search request. It does not define the grammar or semantics of any query that is transmitted. That is, the SEARCH method is a transport mechanism for the query. Different query grammars can be transmitted, each defining its own search semantics. 2.3. The Request The client invokes the SEARCH method on the resource named by the Request-URI. 2.3.1. The Request-URI as Search Arbiter The Request-URI identifies a resource that acts as an arbiter for the search. This resource is the one performing the search but not necessarily the resource that is being searched. That is, there is no implicit relationship between the Request-URI and the search scope. Query grammars MUST explicitly define the relationship between the search scope and the request-URI. Query grammars may, for example, force the request-URI to correspond exactly to the search scope. 2.3.2. The Request Body The client MUST include a text/xml request body containing the DAV:searchrequest XML element. The name of the contained XML element identifies the query grammar being used and defines the criteria, the result record, and any other attributes for the search. 2.4. The Response 2.4.1. 217 Multistatus The search proceeded successfully. The response matches that of a PROPFIND. Each resource listed in the Multistatus response MUST meet the criteria defined by the corresponding search request. 2.4.2. 400 Bad Request The search query was unable to be processed. The search was not initiated. If the server provides a body then the entity will describe the errors involved by using a text/xml entity containing the DAV:searcherror element. 2.5. PROPFIND Response Extensions Query grammars MUST define how the response matches the PROPFIND response. Responses are free to include more information than PROPFIND Reddy, et al [Page 5] INTERNET-DRAFT DASL Friday, March 13, 1998 responses so long as the extra information does not invalidate the PROPFIND response. 2.6. Example This example demonstrates the request and response framework. >> Request SEARCH / HTTP/1.1 Host: ryu.com Content-Type: text/xml Connection: Close Content-Length: xxxxx … the query goes here … >> Response HTTP/1.1 207 Multi-Status Content-Type: text/xml Content-Length: xxxxx http://ryu.com/whales.txt Box type A J.J. Dingleheimerschmidt 259 text/plain HTTP/1.1 200 OK 3. The DAV:searchrequest XML Element Reddy, et al [Page 6] INTERNET-DRAFT DASL Friday, March 13, 1998 The value of DAV:searchrequest is a single element that defines the query. The type of the query is identified by the element name. The value of that element defines the query itself. For example, the following XML document shows a simple natural language query. The type of the query is FOO:natural-language query. The actual query is “Find good Thai restaurants in Los Angeles”. Find good Thai restaurants in Los Angeles 4. The DAV:searcherror XML 4.1. DAV:searcherror DAV:searcherror allows the server to indicate the errors that prevented a search from being executed. DAV:seacherror is a list of elements. Each element defines an error. The name of each element uniquely identifies the error, the value of the element provides any further detail about the error. This draft defines a few simple error responses. Servers may define any extended set of errors and provide that in the error response. 4.2. DAV:too-complex DAV:too-complex indicates that a query was too complex for the server to process. 4.3. DAV:invalid-operator DAV:invalid-operator indicates that a server could not handle an operator used in the query. DAV:invalid-operator contains a single XML element that identifies the operator that is invalid. 4.4. DAV:invalid-scope Reddy, et al [Page 7] INTERNET-DRAFT DASL Friday, March 13, 1998 DAV:invalid-scope indicates that a server cannot search for the named scope. DAV:Scope contains the URI identifying the scope. The optional DAV:arbiter element contains a URI that identifies an arbiter that will be able to handle the query for that scope. Multiple DAV:arbiter elements can be used to show that multiple arbiters cans search on that scope. 4.5. DAV:invalid-type DAV:invalid-type indicates that a type specified for a value in the query was not recognized and caused the query to fail to execute. DAV:invalid-type contains a single URI that identifies the type in question. 4.6. DAV:invalid-grammar DAV:invalid-grammar indicates that the query grammar used in the request is not supported by the server. 5. Discovery Mechanisms 5.1. Query Grammars Clients can determine which query grammars are supported by an arbiter by invoking OPTIONS on the search arbiter. If the resource supports SEARCH, then the DASL response header will appear in the response. The DASL response header lists which grammars are supported. Consult the section on the OPTIONS method and the DASL response header for more information. 5.2. Query Grammar Schema Discovery The DASL response header provides means for clients to discover the set of query grammars supported by a resource. This alone is not sufficient information for a client to generate a query. For example, the DAV:simplesearch grammar defines a set of queries consisting of a set of operators applied to a set of properties and values, but the grammar itself does not specify which properties may be used in the query. Moreover, while the DAV:simplesearch grammar defines a minimal set of operators, it is possible that a resource might support additional operators. Search schema discovery allows a client to discover such information. [Saveen: need to figure out what that minimal set is] Reddy, et al [Page 8] INTERNET-DRAFT DASL Friday, March 13, 1998 [Saveen: Can we use a different word than “schema” – in the context of searching it is a loaded term.] The schema for a given query grammar on a resource is stored as a property of that resource. The name of the property is the same as the name of the query grammar itself. The property value is a queryschema XML element. The contents of this element depend upon the particular query grammar. 6. The OPTIONS Method 6.1. Overview The OPTIONS method allows the client to discover if a resource supports the SEARCH method and to determine the list of search grammars supported for that resource. 6.2. Request The client issues the OPTIONS method against a resource named by the Request-URI. This is a normal invocation of OPTIONS defined in [RFC2068]. 6.3. Response If a resource supports the SEARCH method, then the server MUST list SEARCH in the OPTIONS response as defined by [RFC2068]. DASL servers MUST include the DASL header in the OPTIONS response. This header identifies the search grammars supported by that resource. 6.4. Example This example shows that the server supports search on the /somefolder resource with the following query grammars: http://foo.bar.com/syntax1 and http://akuma.com/syntax2. >> Request OPTIONS /somefolder HTTP/1.1 Connection: Close Host: ryu.com >> Response HTTP/1.1 200 OK Date: Tue, 20 Jan 1998 20:52:29 GMT Connection: close Accept-Ranges: none Allow: OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, COPY, MOVE, MKCOL, PROPFIND, PROPPATCH, LOCK, UNLOCK, SEARCH Public: OPTIONS, GET, HEAD, POST, PUT, DELETE, TRACE, COPY, MOVE, MKCOL, PROPFIND, PROPPATCH, LOCK, UNLOCK, SEARCH DASL: Reddy, et al [Page 9] INTERNET-DRAFT DASL Friday, March 13, 1998 7. New Headers 7.1. The DASL Response Header 7.1.1. Overview The DASL response header indicates server support for query grammars in the OPTIONS method. The value of this header is a list of URLs. Each URL indicates support for a query grammar. 7.1.2. Syntax DASLHeader = "DASL" ":" URL-List URL-List = 1#Coded-URL ; defined in section 8.5 of [WEBDAV] 7.1.3. Example DASL: 8. The DAV:simplesearch grammar 8.1. Introduction DAV:simplesarch uses an extensible XML syntax that allows clients to express search requests that are generally useful for WEBDAV scenarios. DASL-extended servers MUST accept this grammar. DAV:simplesearch has several major components: DAV:select provides the result record definition DAV:from defines the scope DAV:where defines the criteria DAV:sortby defines the sort order of the result set 8.1.1. Example This query will retrieves the content length values for all resources located under the server's "/container1/" URI namespace whose length exceeds 10000. /container1/ infinite Reddy, et al [Page 10] INTERNET-DRAFT DASL Friday, March 13, 1998 10000 8.2. DAV:select DAV:select defines the result record. This document defines two possible values: DAV:allprops and DAV:props, both defined in [WEBDAV]. If the value is DAV:allprops, the result record for a given resource includes all the properties for that resource. If the value is DAV:props, then the result record for a given resource includes only those properties named by the DAV:props element. Each property named by the DAV:props element must be referenced in the Multistatus response. The rules governing the status codes for each property match those of the PROPFIND method defined in [WEBDAV]. 8.3. DAV:from DAV:from defines the query scope. This a list of one or more DAV:scope elements. Each scope element contains a mandatory DAV:uri element and an optional DAV:depth element. DAV:uri indicates the URI for a collection to use as a scope. DAV:depth has the value of “0” or “infinite” indicating non-recursive and recursive searching of a scope, respectively. 8.4. DAV:where DAV:where defines the criteria for inclusion in the result set. The value of the this element is any XML element that defines a search operator that evaluates to True, False, or Unknown. The search operator is applied to the resource as a criteria. If and only if the result of the operation is True, then the resource is included as a member of the result set. Reddy, et al [Page 11] INTERNET-DRAFT DASL Friday, March 13, 1998 8.4.1. Example The example shows a single operator (FOO:dummy-operator) applied in the criteria. This example shows how the grammar can be extended by using XML elements to define operators. In this case an application-specific operation is specified that DASL has not defined. “Hello World” 8.4.2. Example The example shows a more complex operation involving several operators (DAV:and, DAV:eq, DAV:gt) applied in the criteria. This DAV:where expression matches those resources that are “image/gifs” over 4K in size. image/gif 4096 8.5. DAV:sortby DAV:sortby specifies the ordering of the result set. This contains a list of properties for each resource in the result set upon which to sort 8.6. Properties 8.6.1. DAV:rank The rank XML element is a pseudo property whose value is defined only in the context of a query result where the server computes a ranking, e.g. based on relevance. Reddy, et al [Page 12] INTERNET-DRAFT DASL Friday, March 13, 1998 8.7. Search Operators 8.7.1. Introduction This section introduces several XML elements, each defining a search operator for use with the DAV:simplesearch grammar. 8.7.2. DAV:and, DAV:or, DAV:not The DAV:and operator performs a logical AND operation the values it contains. The DAV:or operator performs a logical OR operation the values it contains. The DAV:not operator performs a logical NOT operation the values it contains. The type of DAV:and, DAV:or, and DAV:not is Boolean. 8.7.3. DAV:eq The DAV:eq operator provides simple equality matching on property values. For string values the comparison is case-sensitive. The type of DAV:eq is Boolean. 8.7.4. DAV:lt, DAV:lte, DAV:gt, DAV:gte The DAV:lt, DAV:lte, DAV:gt, and DAV:gte operators provide comparisons on property values. For string values the comparison is case-sensitive. The type of DAV:eq is Boolean. 8.7.5. DAV:contains The DAV:contains operator provides a content-based comparisons on property values or the text-content of a resource. Reddy, et al [Page 13] INTERNET-DRAFT DASL Friday, March 13, 1998 If a DAV:prop is given, then the operation applies to the single property named by DAV:prop. Otherwise, the operation applies to text content of the resource. If DAV:casesensitive is given and it’s value is “t”, then the operation is performed with regard to case. Otherwise if the value if “f”, then the operation is performed without regard to case. The type of DAV:contains is Boolean. 1.1.1.1. Examples The example below matches those text resources that contain the phrase "telecommunications industry". telecommunications industry The example below matches those text resources where the foo:author property contains the word "Smith" with case-sensitivity enabled. Smith t [Saveen: how to indicate that this is done with a particular language? Or do we simply say simple string compare] 8.7.6. DAV:literal DAV:literal allows typed, literal values to be placed in an expression. Typing is specified through the XML-DATA specification. If no typing is specified, then “string” in XML-DATA is assumed. 8.8. Typing Property values may be typed. For example, the typing scheme described by [XMLDATA] may be used. Some operations are valid only in conjunction with typed values. For example a “greater than” operation is valid for those values that can be ordered. 8.8.1. Typing Errors If a server cannot perform an operation because a property of a given type does not support a operator specified in the query, then the server MAY indicate the error in the DAV:searcherror response. Alternatively, server MAY use the other mechanisms to deal with typing errors, such as three-valued logic. Reddy, et al [Page 14] INTERNET-DRAFT DASL Friday, March 13, 1998 8.9. Query Schema for DAV:simplesearch 8.10. Versioning TBD. 8.11. Variants TBD. 8.12. Internationalized Content All string values in the query are assumed to be in the character set used by the containing XML document. If the XML document uses a character set not understood by the server, then the server MUST respond with 400 (Bad Request). 9. Security Considerations This section is provided to detail issues concerning security implications of which DASL applications need to be aware. All of the security consideration of HTTP/1.1 also apply to DASL. In addition, this section will include security risks inherent in searching and retrieval of resource properties and content. 10. Authentication Authentication mechanisms defined in WEBDAV will also apply to DASL. 11. IANA Considerations This document uses the namespace defined by [WEBDAV] for XML elements. All other IANA considerations mentioned in [WEBDAV] also applicable to DASL. 12. Change History Feb, 14 - Initial Draft Feb, 28 Reddy, et al [Page 15] INTERNET-DRAFT DASL Friday, March 13, 1998 - Referring to DASL as an extension to HTTP/1.1 rather than DAV - Added new sections "Notational Conventions", "Protocol Model", "Security Considerations" - Changed section 3 to "Elements of Protocol" - Added some stuff to introduction - Added "result set" terminology - Added "IANA Considerations". Mar, 9 - Moved sub-headings of "Elements of Protocol" to first level and removed "Elements of Protocol" Heading. - Added an sentence in introduction explaining that this is a "sketch" of a protocol. Mar, 11 - Added sortby, datatyping, three valued logic, query schema property, and element definitions for schema for simplesearch. April 8 - made changes based on last week’s DASL BOF. 13. References [DASLREQ] S. Reddy, J.Slein, "Requirements for DAV Searching and Locating", February 1998, internet-draft, work-in-progress, draft-reddy- dasl-requirements-01.txt [RFC2068] R. Fielding, J. Gettys, J. C. Mogul, H. Frystyk, and T. Berners-Lee, "Hypertext Transfer Protocol -- HTTP/1.1", RFC 2068, U.C. Irvine, DEC, MIT/LCS, January 1997. [RFC2119] S. Bradner, "Key words for use in RFCs to Indicate Requirement Levels." RFC 2119, BCP 14. Harvard University. March, 1997. [WEBDAV] Y. Goland, E.J. Whitehead, A. Faizi, S.R. Carter, D.Jenson, "Extensions for Distributed Authoring on the World Wide Web", October, 1997, internet-draft, work-in-progress, draft-ietf-webdav-protocol-06. [XMLDATA] A. Layman, E. Jung, E. Maler, H.S. Thompson, J. Paoli, J. Tiguw, N.H. Mikula, S. De Rose, “XML-DATA”, Jan, 1998, work-in-progress, http://www.w3.org/TR/1998/NOTE-XML-data/ 14. Author's Addresses Saveen Reddy Microsoft One Microsoft Way Redmond WA, 9085-6933 Email: saveenr@microsoft.com Del Jensen Novell 1555 N. Technology Way Reddy, et al [Page 16] INTERNET-DRAFT DASL Friday, March 13, 1998 M/S ORM F111 Orem, UT 84097-2399 Email: dcjensen@novell.com Surendra Reddy Oracle Corporation 600 Oracle Parkway, M/S 6op3, Redwoodshores, CA 94065 Email: skreddy@us.oracle.com Phone:(650) 506 5441 Rick Henderson Netscape Email: rickh@netscape.com Jim Davis Xerox PARC 3333 Coyote Hill Road Palo Alto CA 94304 650-812-4301 Email: jdavis@parc.xerox.com Alan Babich Filenet Email: ababich@filenet.com Expires Oct 3, 1998 Reddy, et al [Page 17]