Web addresses in HTML 5

1 Terminology

A Web address is a string used to identify a resource.

The term "Web address" in this specification is used to include not only Uniform Resource Identifiers (URIs) as they are defined by RFC 3986 and Internationalized Resource Identifiers (IRIs) as they are defined by RFC 3987, but also other strings of characters which can be used to identify Web resources when processed appropriately.

A Web address is a valid Web address if at least one of the following conditions holds:

• The Web address is a valid URI reference (i.e. it matches the grammar for <URI-reference&ft; given in RFC 3986).

• The Web address is a valid IRI reference (i.e. it matches the grammar for <IRI-reference&ft; given in RFC 3987), and it has no query component.

• The Web address is a valid IRI reference and its query component contains no unescaped non-ASCII characters [RFC3987].

• The Web address is a valid IRI reference and the character encoding of the Web address's Document is UTF-8 or UTF-16 [RFC3987].

A Web address has an associated URL character encoding, determined as follows:

If the Web address came from a script (e.g. as an argument to a method)

The Web address character encoding is the script's character encoding.

If the Web address came from a DOM node (e.g. from an element)

The node has a Document, and the URL character encoding is the document's character encoding.

If the Web address had a character encoding defined when the Web address was created or defined

The Web address character encoding is as defined.

2 Parsing Web addresses

To parse a Web address w into its component parts, the user agent must use the following steps:

1. Strip leading and trailing space characters from w.

2. Percent-encode all non-URI characters in w.

   This probably needs to be laid out in more detail.

   Note: this step will replace all of the following characters with a percent-encoded equivalent:

   • all characters with codepoints less than or equal to U+0020 (i.e. the C0 control characters)
   • all characters with codepoints greater than or equal to U+007% (i.e. U+007?F and all non-ASCII characters in the w)
   • U+0022 double quotation mark
   • U+0025 percent sign
   • U+003C less-than sign
   • U+003E greater-than sign mark
   • U+005C reverse solidus (backslash)
   • U+005E circumflex accent
   • U+0060 grave accent
   • U+007B left curly bracket
   • U+007C vertical line
   • U+007D right curly bracket

   As a result of percent-encoding the percent sign, any occurrences of percent-encoding in the Web address will be double-encoded at this step.

3. If w begins with either of:

   • a string matching the <scheme> production, followed by "://"
   • the string "//"

   then percent-encode any left or right square brackets (U+005B, U+005D, "[" and "]") following the first occurrence of "/", "?", or "#" which follows the first occurrence of "//".

   Otherwise, percent-encode all left and right square brackets.

4. Percent-encode all occurrences of U+0023 (Number sign, "#") after the first.

5. Parse w using the grammar in RFC 3986.

6. If w doesn't match the <URI-reference> production, even after the above changes are made to it, then parsing the Web address fails with an error. [RFC3986]

   Otherwise, parsing w was successful; the components of the Web address are substrings of w defined as follows. First, the substring of the modified w which matched a particular production in RFC 3986 is identified; then any percent-encoded characters in that substring are decoded. The resulting string (called here the "decoded substring) is one of the named components of w.

   As in the algorithm previously given, Web addresses containing percent-encoded characters here have components which similarly contain percent-encoded characters.

   <scheme>

   The decoded substring matched by the <scheme> production, if any.

   <host>

   The decoded substring matched by the <host> production, if any.

   <port>

   The decoded substring matched by the <port> production, if any.

   <hostport>

   If there is a <scheme> component and a <port> component and the port given by the <port> component is different than the default port defined for the protocol given by the <scheme> component, then <hostport> is the decoded substring that starts with the decoded substring matched by the <host> production and ends with the decoded substring matched by the <port> production, and includes the colon in between the two. Otherwise, it is the same as the <host> component.

   <path>

   The decoded substring matched by one of the following productions, if one of them was matched:

   • <path-abempty>
   • <path-absolute>
   • <path-noscheme>
   • <path-rootless>
   • <path-empty>

   <query>

   The decoded substring matched by the <query> production, if any.

   <fragment>

   The decoded substring matched by the <fragment> production, if any.

   <host-specific>

   The decoded substring that follows the decoded substring matched by the <authority> production, or the whole string if the <authority> production wasn't matched.

3 Resolving Web addresses

To resolve a Web address to an absolute Web adddress relative to either another absolute Web address or an element, the user agent must use the following steps. Resolving a Web address can result in an error, in which case the Web address is not resolvable.

1. Let w be the Web address being resolved.

2. Let encoding be the character encoding of the Web address.

3. If encoding is UTF-16, then change it to UTF-8.

4. If the algorithm was invoked with an absolute Web address to use as the base Web address, let base be that absolute Web address.

   Otherwise, let base be the base URI of the element, as defined by the XML Base specification, with the base URI of the document entity being defined as the document base Web address of the Document that owns the element. [XMLBASE]

   For the purposes of the XML Base specification, user agents must act as if all Document objects represented XML documents.

   It is possible for xml:base attributes to be present even in HTML fragments, as such attributes can be added dynamically using script. (Such scripts would not be conforming, however, as xml:base attributes are not allowed in HTML documents.)

   The document base Web address of a Document is the absolute Web address obtained by running these substeps:

   1. Let fallback base url be the document's address.

   2. If fallback base url is about:blank, and the Document's browsing context has a creator browsing context, then let fallback base url be the document base Web address of the creator Document instead.

   3. If there is no base element that is both a child of the head element and has an href attribute, then the document base Web address is fallback base url.

   4. Otherwise, let w be the value of the href attribute of the first such element.

   5. Resolve w relative to fallback base url (thus, the base href attribute isn't affected by xml:base attributes).

   6. The document base Web address is the result of the previous step if it was successful; otherwise it is fallback base url.

5. Parse w into its component parts.

6. If parsing w resulted in a <host> component, then replace the matching subtring of w with the string that results from expanding any sequences of percent-encoded octets in that component that are valid UTF-8 sequences into Unicode characters as defined by UTF-8.

   If any percent-encoded octets in that component are not valid UTF-8 sequences, then return an error and abort these steps.

   Apply the IDNA ToASCII algorithm to the matching substring, with both the AllowUnassigned and UseSTD3ASCIIRules flags set. Replace the matching substring with the result of the ToASCII algorithm.

   If ToASCII fails to convert one of the components of the string, e.g. because it is too long or because it contains invalid characters, then return an error and abort these steps [RFC3490].

7. If parsing w resulted in a <path> component, then replace the matching substring of w with the string that results from applying the following steps to each character other than U+0025 PERCENT SIGN (%) that doesn't match the original <path> production defined in RFC 3986:

   1. Encode the character into a sequence of octets as defined by UTF-8.
   2. Replace the character with the percent-encoded form of those octets. [RFC3986]

   For instance if w was "//example.com/a^b☺c%FFd%z/?e", then the <path> component's substring would be "/a^b☺c%FFd%z/" and the two characters that would have to be escaped would be "^" and "☺". The result after this step was applied would therefore be that w now had the value "//example.com/a%5Eb%E2%98%BAc%FFd%z/?e".

8. If parsing w resulted in a <query> component, then replace the matching substring of w with the string that results from applying the following steps to each character other than U+0025 PERCENT SIGN (%) that doesn't match the original <query> production defined in RFC 3986:

   1. If the character in question cannot be expressed in the encoding encoding, then replace it with a single 0x3F octet (an ASCII question mark) and skip the remaining substeps for this character.
   2. Encode the character into a sequence of octets as defined by the encoding encoding.
   3. Replace the character with the percent-encoded form of those octets. [RFC3986]

9. Apply the algorithm described in RFC 3986 section 5.2 Relative Resolution, using w as the potentially relative URI reference (R), and base as the base URI (Base). [RFC3986]

10. Apply any relevant conformance criteria of RFC 3986 and RFC 3987, returning an error and aborting these steps if appropriate. [RFC3986] [RFC3987]

    For instance, if an absolute URI that would be returned by the above algorithm violates the restrictions specific to its scheme, e.g. a data: URI using the "//" server-based naming authority syntax, then user agents are to treat this as an error instead.

11. Let result be the target URI (T) returned by the Relative Resolution algorithm.

12. If result uses a scheme with a server-based naming authority, replace all U+005C REVERSE SOLIDUS (\) characters in result with U+002F SOLIDUS (/) characters.

13. Return result.

A Web address is an absolute Web address if resolving it results in the same Web address without an error.