URI Schemes and Web Protocols

Draft Tag Finding 12 June 2005

This version:
Latest version:
Previous version:
Noah Mendelsohn, IBM Corp. <Noah_Mendelsohn@us.ibm.com>

This document is also available in these non-normative formats: XML.


This document discusses the deployment of protocols and URI schemes for use on the World Wide Web. Implications for correct configuration of user agents, servers and proxy gateways are discussed. Guidelines are also provided for deciding whether a new protocol or scheme is merited, and for maximizing interoperability of new protocols with those that are already widely deployed.

Status of this Document

This document is an editors' copy that has no official standing.

This document has been produced by the W3C Technical Architecture Group (TAG). This finding addresses TAG schemeProtocols-49.

This version of the document is a very preliminary sketch of a possible finding. Essentially, it is a snapshot of the editor's work-in-progress, made available so that the TAG will have something to discuss informally at the 14 June 2005 Face to Face meeting.

This document builds on and complements information from [RFC 2717] and [RFC 2718]. At the time of this writing, an Internet Draft has been submitted that would revise and subsume both of those RFCs (see [RFC 2717bis]). Although such drafts are not suitable for normative reference, this finding is intended to be consistent with the directions signalled in those revisions. When and if a revision to the RFCs becomes accepted, the TAG intends to republish this finding with the appropriate references and with any necessary changes to content.

Editorial note 
Is this the right way to handle the reference to 2718bis2718bis?

Additional TAG findings, both accepted and in draft state, may also be available. The TAG may incorporate this and other findings into future versions of the [AWWW].

The terms MUST, SHOULD, and SHOULD NOT are used in this document in accordance with [RFC 2119].

Editorial note 
Should a finding like this actually make use of the formal rfc2119 terminology?

Please send comments on this finding to the publicly archived TAG mailing list www-tag@w3.org (archive).

Table of Contents

1 Preface
2 Terminology
3 URI Assignment and Protocols
4 Gateway Proxies
5 Selection of Protocols by User Agents
6 Protocol design: consistency of operations and formats (To Be Supplied)
7 References

1 Preface

Of the many ways in which the Web can be extended, the provision for new URI schemes may be the most fundamental. URI schemes are often created to signal the use of new protocols for transfer of resource-related information. Embodied in such protocols are operations, such as GET, POST and DELETE in the case of HTTP, which determine the sorts of interactions that are possible with a given resource. The operations in turn determine the format and typing of information exchanged on the Web.

Precisely because so many aspects of resource naming and interaction are subject to change through introduction of URI schemes, new schemes can undermine the interoperability of the Web. [AWWW] explains why unnecessary proliferation of URI schemes must be avoided: "While Web architecture allows the definition of new schemes, introducing a new scheme is costly. Many aspects of URI processing are scheme-dependent, and a large amount of deployed software already processes URIs of well-known schemes. Introducing a new URI scheme requires the development and deployment not only of client software to handle the scheme, but also of ancillary agents such as gateways, proxies, and caches. See [RFC 2718] for other considerations and costs related to URI scheme design...Because of these costs, if a URI scheme exists that meets the needs of an application, designers should use it rather than invent one."

Conversely, the introduction of new protocols, operations, and data formats may sometimes be essential if the Web is to continue as a universal information space integrating the broadest possible range of resources. Already a wide variety of information is being shared through peer-to-peer protocols, many of which are not well integrated on the Web, and there is reason to believe that the highest quality multimedia feeds may not be best distributed and controlled through HTTP. Immersive user interfaces may require interaction protocols which are more flexible than those in in widespread use on the Web today, and so on. For such reasons, it is important to explore the tradeoffs involved in deploying new schemes, protocols and operations. This finding attempts to provide useful guidelines for the introduction and use of URI schemes and their associated protocols.

[RFC 2718] sets out guidelines and caveats for the creation of new URI schemes. This finding provides complementary information relating to the deployment of resources using such schemes, the design of protocols, the choice of operations to be supported, and the suggested behaviour of user agents. Indeed, this finding for the most part avoids restating principles already covered in [RFC 2717] and [RFC 2718]; readers are encouraged to become familiar with them before proceeding.

2 Terminology

This finding is intended to cover a broad range of information formats and protocols including client/server, peer-to-peer, streaming multimedia, multicast, etc. For convenience, the following client/server-oriented terminology is used except in cases where the extension to other protocols is unlikely to be clear:

User agent
A programmed entity, typically software, that uses a protocol to retrieve or update the state of a remote resource.
A programmed entity, typically software, that maintains the state of a resource. Servers implement network protocols that support requests to retrieve (representations of) and/or to update the resource's state.
In the case of HTTP, state is exchanged in the form of media-typed resource representations, but other protocols may implement other abstractions or use different formats or typing systems.

3 URI Assignment and Protocols

For schemes such as http and ftp, the association of a URI to a resource is defined in terms of the corresponding protocol. Thus, the resource identified by http://example.org/resource1 is by definition the one for which representations are returned (GET) or updated (PUT) when that URI is supplied as the HTTP Request-URI (see [RFC 2616]). Unless otherwise stated, this finding deals only with such protocol-associated URI schemes.

Subtleties arise when such URIs are employed without deploying a server for the resource. For example, it is common to use XML namespace names based on the http scheme even when no server is providing representations for that namespace. Deploying such a server is desirable, but is not required by Web architecture. When there is no such server, the URI chosen SHOULD be consistent with eventual server deployment. So, in the case of HTTP, it is inappropriate to base a URI on a DNS name that is not registered, because the DNS name might later be assigned to an organization that would use it for a purpose inconsistent with serving representations of the resource. Similar considerations apply for other schemes and their associated protocols.

4 Gateway Proxies

In the simplest case, the protocol associated with the URI directly connects the user agent to the resource provider.

Picture of direct connection to resource provider.

The Web also allows for gateway proxies, which convert from one protocol to another. In such cases, the server offers the resource using one protocol, but the user agent access it through another. For example, HTTP can be used as a proxy for the FTP protocol.

Picture of proxy/gateway connection to resource provider.

The following considerations apply to the implementation of gateway proxies:

On the Web, URI names are typically used "on the wire" as the means by which protocols identify resources to be accessed. HTTP, for example, uses a URI as its Request-URI. When one such protocol is used as a proxy to another, the two "hops" may or may not use the same URI. When they are not the same, then there are two URI's identifying the same resource. (Strictly speaking, the two URI's name the resource and the proxy of the resource respectively, but for many practical purposes the effect is similar to having two names for the resource itself.) [AWWW] explains the disadvantages of assigning more than one URI to a single resource. For those reasons, protocols intended for use with gateways SHOULD be designed to avoid the requirement to generate such duplicate URI names. HTTP, for example, provides for the use of non-http scheme URIs as Request-URIs; accordingly, the same URI can often be used on both "hops". Conversely, URI duplication may be unavoidable when the gateway protocol demands naming with a particular scheme.

Editorial note 
Does this section us an appropriate mix of RFC 2119 "MUST"s and "SHOULD"s vs. more informal guidance?

5 Selection of Protocols by User Agents

This section discusses the means by which a user agent can select an appropriate protocol for accessing a resource.

The specification for a URI scheme determines the normative association of URIs from that scheme to resources. For protocol-based URIs, that association is typically defined in terms of the protocol (see 3 URI Assignment and Protocols). In such cases, a user agent can determine a protocol based on inspection of the URI, and in the common case where there is one protocol associated with a scheme, the scheme name directly determines the protocol. It is, for example, always acceptable for a user agent to attempt an HTTP connection to a resource named with the http scheme.

The means by which user agents determine that a gateway protocol is to be used are specific to each user agent. Using the example above, a user agent would require some configuration to indicate that ftp-scheme resources were in fact to be accessed using the HTTP protocol. This is similar to the other sorts of proxy configuration that are commonly required of Web browsers.

6 Protocol design: consistency of operations and formats (To Be Supplied)

This section has not been written. The paragraphs below are placeholders with reminders of possible topics to be covered.

To be supplied: explain that it's much easier to support new protocols in a user agent if the operations of that protocol are even generally similar to those of HTTP or other widely deployed protocols. So, a high def streaming video protocol may not support exactly an HTTP "Get", but if it supports something in the same spirit then a browser can probably provide a fairly consistent navigation experience as one goes from a web page to a movie and back.

To be supplied: similarly, if a peer-to-peer protocol supports retrieval of media typed octet streams, then browsers can use existing renderers, caches, etc. This will link to the AWWW GPN on reusing formats.

To be supplied: operations on the wire vs. operations at the endpoint. In HTTP, GET is visible both as a browser operation and on the wire. In peer to peer, you might have a very compatible operation at the browser that turned into all sorts of strange traffic on the wire. That's still a good thing to go for: if you can simulate as much of the HTTP "endpoint API" as possible, then you get a lot of browser compatiblity, even if the on the wire protocols are radically different.

7 References

I.Jacobs, N. Walsh, Architecture of the World Wide Web. W3C. December, 2004. (See http://www.w3.org/TR/webarch/.)
RFC 2119
S. Bradner. Key words for use in RFCs to Indicate Requirement Levels. IETF. March, 1997. (See http://www.ietf.org/rfc/rfc2119.txt.)
RFC 2717
R. Petke, I. King Registration Procedures for URL Scheme Names. IETF. November, 1999. (See http://www.ietf.org/rfc/rfc2717.txt.)
RFC 2616
R. Fielding, J. Gettys, J. Mogul, H. Frystyk, P. Masinter, P. Leach, T. Berners-Lee Hypertext Transfer Protocol — HTTP/1.1. IETF. June, 1999. (See http://www.ietf.org/rfc/rfc2616.txt.)
RFC 2717bis
T. Hansen, T. Hardie, L. Masinter Guidelines and Registration Procedures for new URI Schemes. February, 2005. (See http://ietfreport.isoc.org/all-ids/draft-hansen-2717bis-2718bis-uri-guidelines-03.txt.)
RFC 2718
L. Masinter, H. Alvestrand, D. Zigmond, R. Petke Guidelines for new URL Schemes. IETF. November, 1999. (See http://www.ietf.org/rfc/rfc2718.txt.)
RFC 3986
T. Berners-Lee, R. Fielding, L. Masinter Uniform Resource Identifier (URI): Generic Syntax. IETF. January, 2005. (See http://www.ietf.org/rfc/rfc3986.txt.)
--=_mixed 0054255B8525701E_Content-Type: application/octet-stream; name="ProxyConnect.png" Content-Disposition: attachment; filename="ProxyConnect.png" Content-Transfer-Encoding: base64 iVBORw0KGgoAAAANSUhEUgAAAqAAAAEMCAMAAAAs1PT/AAAAAXNSR0ICQMB9xQAAAadQTFRFAAAA Bw8JFBYWHh4fGh4bDg8PBg8JGh0eEBAQCAgIHi0iHC0hHTwkHTUjIB4gMjIyPDw+IDwnKjUtND02 MjQ1IS0kLzM0IDUlMTw0JTUpMzQ2JzUrKSwtMzk7Ojs9IzwpKiwtNDo7LjM0MDAwMzOZJFIvIkss O0Q9P0hJOkFCK3c9Inc2KGE1KGk4K3A8JH45Ln9BLYVCMJ1KL41FKpRDM6ROM7JRNbpUN8FYO9Be OclbPthiQD1ARkZGWlpaQ0xGRU9QQ0NGTFROS0tOVFJWUV1fU1tVQEBAWFRYXGNeVmNlXXBxQN9l RO5sQudpbm5ua2tsdnl4f3l/bGpuZmtofHh9YHFzY3d4aX5/ZGJmbXJvfXl+a36AfYB/fJOVcYWI gICAkZGRlo+WkJCRiIiJgpqcjYiNlY+VjY2NhqGjjKiqiKGknr2/mLa4kq+xpp6mo56joKCgq6ar v7a+rqautq62q6urr6euv7+/r9LVtdncqcvOpMTHu+Djx77Gx8cA1NakwMDA3Nzcz8XO39Xe183W 0tLS59zm//P+5ubm9+v27+Tu////LXpUGwAAAAlwSFlzAAAOxAAADsQBlSsOGwAAABl0RVh0U29m dHdhcmUATWljcm9zb2Z0IE9mZmljZX/tNXEAAA7ESURBVHja7d0Jg9vEAYbhKZRCw5GQhKMcIYCg nG0TCkmANOEqhcbhpqVNwVxJCRByQNsEtCDvhpV3/aO7tlfSjC7rmBkdfj/Cem1LWo/n8Wg0OiwE IYSQyvEJaWluASgBKCEAJQAlBKCEAJQAlBCAEoASAlBCAEoASghACUAJASghACUAJQSgBKCEAJQQ gBKAEgJQAlCAEoASAlACUEIASghACUAJaQXQYZXw1hNLQIfjCvEQSuwAreRzPAYoASgBKEAJQAlA AUoA2iOgFyaTnRdKZedk0iM4tsqvD+gbxeL1BOjE8yblstO70COgtsqvEejRQukTUK9U+gfURvmt Aw2EApTyAxSgAAUoQAEKUIACtEIFjYIc9bz87+sd9hKohfIDtE4FBWXaqiAxGU6GOf9EH4FaKD9A 61WQd9RbaqDGyw9QgAK016v45QZ6FKDtrqDhkgMdAhSgAAUoQAEKUIACFKAABShAAdqZCmKYCaBt ryDPW26gZssP0FoV9Kd5lhao+fIDtF4Fffyx5328xECNlx+gAAVoz4F6Sw7UAyhAAQrQShXERhIb SS2uIOmUByGGYpjzr5dALZQfoLVakKhUS3nSnIXydxboxUv63uw31ytWkPWzOlc/0VfqT1ZrA+Ws zsy89Yu39VWV2LHSDaDfXLNLX6l3XfMNQA0B3Tj22OM6gT5366ddAPrO8S90Av3i+DsANQJ0bffT A61AByefeLdCBYWxAnTzhVPuWZ1Az7qnXtisA9RG+bsI9NKNhweagQ4GB/aUqyulgoQFoFf3funq Bup+ufeqDqACoFI+u+nEQD/QwTO3ldpomK3iwgtnmAf67e1fufqBul/d/m11oDbK3zmgGy8+OeWk H+jgyM3ny1aQvT7oe49cdk0AdS8/8l51oPRBE1nffXBgCOjg5EMlOqLxKwwLk0A3X3rfdc0Add33 X6rSEbVV/o4BXdlxeGAMaKmO6KyCRlFMAr2693PXHFD38yodUVvl7xbQT299bmAS6FZHtGhdzVZx dlrQ72bdT3NAtzqi31UC6vWxBf1pVBnoxmtPnAwpPf7829oSAR0cueNiiQqSLzBoDOjpRy+7IdBr 9ZX62hCoe/nR05WAWii/xq+hGf1U6L/KX6KwfudTEaTBicf15Y/ych9+q3gFWdhI2Hj5dTfKlb/9 VVv+Li3Xff3ljQpAO7WRZPp7klZ/c2hgJY8d26xSQWYu4Lp214eulXx411o9oK2/gK1hoOdvPjKw lKfvvlq+gswcD3npnq9dS/n6nlKH39gpf3eAvvvQyYG1PFugI2qlgk7fe9m1lsv3lumI9hboj1WA bu45MLCZEw//qwUVpHY/LaRMR5QWVMrqbc+EdkRwk+znBA8FUyr3CiWa+HcvbjRdQev7Yt3P7QK5 YYHDR9TnSyQ28Yf7Ch8Z2yGgnmmgF+87MkgCVUCJ5HOxmxSEeUAHB3evNVtBK/vP5WgS8iOpN1kI c587t3+ld0CrCR0WB/rWwycG9oEODt94qckKOnP/9659oO7395/pHVB/WCGFF7557LE0Q6WATlf+ 81W++jNxq+A9cdNnjVXQxgevXHGrA52u/OerfPVn4jaJ98orH2z0DWhBr5UWtnb304P6QEN8Yn4r /564jfLknxuqoM19Hy1o7hYADfGJ+a38e+I2lo/2VRkH7hBQf5j2PfEVj15a3REbnhfytk8caMZG khjkAR1kAj3w241mKmgtPDokuZGkAM3YSBJuHlA3H+jne4sM2XcZqK+t/fSnOzgPlm1BEx3KakDz j74zW0Hh8XWFWtDE83WAFjz6rptAf16c0ssMjlC2DfS5W883WUHvHb/SBNArxwsev9xRoAs3jn6u sNT5OR6WgR5acCqy8Qr65oEf7AP94YGipyLTgkqZnSVnDGjqRtJTd643XUGr+8+aAZq9kXR2f+Hz sjo0UF8jRf/G2u6DtYGGQ0kFhplOPvHaRvMVtLnrVB2g4VBS4WGmU7uKn/zRGaDDein4V2IdUcMp ch0HKxX0TrIjai5XSl3HoStA6/EsscNT6YiazaEiV8KxU0HJjqixFO9+dgxobBdmcqdm9iNlgPqX 7njWjs+F3U+bFZToiJpKie5n94CORiMVX9BAZgD9cbzdfpY6pumqslPp5DP6clhe7uLup9UKinVE z+rLuardz84BHW1hC081mh3w+cZweibqMLsF3ZphVBaoulv+97/coy2iXPczvYKMXcBW7oievW6X tlx3rmL302r5NQCd+pSAjosB/XfJPugs0oFNZk47PrSj6uFmBq8sInVEzZx2XLL7abX8mlrQUQR0 Cu+NYfQzA+h/qgD1L4YdUSNAD95Z+YBdk5e+WQ0PDTUC9Nz+Cpey7RLQcdQFDXqf86tN5ADdesGl V/HThB1RE0CffLH6KQ9Gr80UHt1kAmixo5c6DTSxha6OJWUBrdCCTjuifzAENP/4z0aBhseH6gda 9PjP7gLNy0xhxjMVgfr+P2YneGoHuuAI+maBBkfYawda+Aj6fgLVOFAvZXaKvG6gi85Bahro/Bwl 3UCLn4PUXaC2BuqlTM/y1Ay0TPezGaD++lZHVDPQj/atV52bFjQvm3sOaAVarvvZENBpR1TvlyhU 7H52C+i4Rgta40Lg7/5KJ9Aby3/pUiNfQ3Pmep1Arz9TY+6uAjV9CbEw5/+rr6reXCs/TzNf5LVy Wl+p/7lSZ26AtjzNAF2+8gMUoABdBqBL+W3HFsoPUF0VtIzfF2+h/AAFKEABClCAAhSgAAUoQKkg yg9QgC4L0HpCu+sToC0GOjQTKojyawE6BChAWw10bCbdEgrQdgMdza8mEjseYDySk/OsOuH8rFCA Un5dQIejrZc0Svj0Rjn3lPuj2JSzMz2pIMpvE+jIAyhA7QOdruGH2+vl3DV8Amh8vS6v4gEKUM1A yxyrOk50PMdq/3N2nlKnhAK0xUCH45FXCuj29cVGqav4bjahAO0V0O1WU+p0qlgBClDNQL1yK/n5 ij4d6JR754F6+ac8iN6fk2So/KWBzrugXpUsAtopoZw019KT5iSgBbeUMkbnlTGnDgK9cGHrX5ns vNAroJbKXwXocFy0BVUkjsbKriSv451QUSE9Amqr/GWBzhq6okBHca5ej9bxAG0v0JGnrOJH+UDD RhSgALUENBhhz9hlFOt9pu5BSozaAxSg9YEO49f31plOH2FPzKQCUJOhQghASZ+BTkwGoKQmUMOh Qkg9oOOfhwb/owUl9YAa7oTSgpKaQAkBKCEAJQAlBKAEoIQAlBCAEoASAlDSWCwffA1QUhJo7Bag BKCEVAEa4Indiu3nhT+9G58IoMQSULHoVtXqV2l2AUpKAo02khR7fjZU5UmAErNAI2iRVFEAaLVt f4CSCkAT7WHq2jzZgpZPHaCi0F+G/zIAze2LthqoyH++G3wLjk0HG7a9/kxKtZ63kSQUoM1sJC0R 0GIvVijtS6dKWPq9yBtmmt5Kb0Zjw0wi85UKX4gY0O3XLE8hqr7olgIVqeszuji1ogdo6gBYpNdP tvpBp7prLag89hx88qR3QWk6pY+gNEP0DojUFpeYAuonB8ZUoJn96C4BjRdG+BlA1YnCKeWf8szE JNCwGRFyu+D7SuWF7YboHtDE2LSvaItJFCK2u0XIhEVibww+jQOVmxb5HRdZIDvZgiaAylT9xBsQ lVx6FyKg0tM4NA80fRsh2YL2EKjakPqS2aTbRLu7aJgDoDW2o+X2I5thVCVd74P66av4dKCJPqgK NL4MYgBo1NtKO94qVl/xYabo4c4CVRpBeXMnPm6hfpKVpzsN1MIoodV98Z2tiRjQxBCR9KEU8oc2 /AiGH2BJuRBV96606m0x/PJtAe14VZj85HX9Y2v29VtrQXu9i3qpgUq1K8Ium0jbq9hqoD1OrTew w+9+REfePygPbKh7FQFKmkDqpxyxpG+cBqCkrtHk/sHEYwAljQKNfvXlYWHfByhpUiZASYuBpu9J TDuSHqCkGaLqnsTY4UNCw5EwACWtDkAJQAkBKAEoIQAlBKAEoIQAlACUEIASAlACUEIASgBKCEAJ ASgBKCEAJQAlBKCEAJQAlBCAEoAClAC0ci6RVmQFoOkRpBW5BaAZQB/8H2k+DwI0C+irE9J8XgUo QAEKUAJQgAIUoASgACUABShAAUoAClCAApQAFKAEoLWBBgcwFHtvRbkFiMILCuZ3nJSJne3/S8TR ONXWi5oHoE0ALfXeiuyHRKllivS76RQbB1pucoAuGdAKPotN75RcGECbBCqCn9vrazGZ3Sp3ZhOJ 6Nc4UBHMImZzyI8llh51EMIHo3WpM5mvUR0F6Hw16wRYnGCa+cPynfk80lpZmnY+aXyitIVFD0qf FPUpZRJH+gPKRADV0YJuuxGhpEnwQ7qNnk9pQYV0K9TFBUDlSSaT+F2JTdyFgiC8mTuVZnCiJx2p 0VOnnSjT5SxMfjJj+tT5Yw8CtA7QaBtHKM2pSADKACovQAKahja29BygMVzORG1OnUnSTBxMKtBJ Enr+BEFTnWgp0+dzJikLBaiWFjRSE62U04DGttklsMFTCtDEY9IqvyTQcFPakdfbmUClLe8cf07O BE78L0QLzZhPBZq+7Q/QekBFgRY0dSa1OxqjJ63ea7Wg0taKU6AFXdhAZq6V481koj1Mnc8psFEF 0IpAReq6WS/Qon3QXKDOxFm4Vi4ONAPYQqC5fVGA6gcq1C0cFejCjaRFfVDphyi1keQkhEgtZ3IV 7xTrg25vx+cAS24k5bWY6S+GPqgGoME2jjwQlNKClhhmmoSLkx6T+qmyyNjCtgdy5NEiJ7XPOYlG i2ImksNM0u9hRzLc0MoYLkqZK3eYSXkxDDNpBFosYmI3jvEZnUljAShAF08DUIC2tgXVeOAHQNsB lAAUoAAFKDoAClACUIACFKAEoAAFKEABClCAEoDWB3rDq6T53ADQjDxPWpG/AJQQgBKAEgJQAlBC AEoIQAlACQEoASghACUEoASghACUABSgBKCEAJQAlBCAEqIA5Zwb0tb8WhDS6vwfoDHimDTiTyMA AAAASUVORK5CYII