Re: [whatwg/url] Enumerate differences with URI & IRI in an appendix (PR #923)

@domenic commented on this pull request.

This would probably also benefit from some review from RFC partisans.

I suggested one paragraph about expanding the model differences, but I still feel there's more to say. But, perhaps your intent is to focus more on concrete mismatches and deemphasize that aspect.

> + <code>sm%F6rg%E5sbord</code>.
+
+ <li><p>Leading and trailing <a>C0 control or space</a> are stripped from the input; interior
+ U+0009, U+000A, and U+000D are silently removed.
+ <code>https://exa&#x9;mp&#x0A;le.com/</code> (with a U+0009 between "<code>exa</code>" and
+ "<code>mp</code>" and a U+000A between "<code>mp</code>" and "<code>le.com/</code>") →
+ <code>https://example.com/</code>.
+
+ <li><p>Out-of-grammar code points in the path, query, or fragment are percent-encoded rather
+ than rejected, and a bare U+0025 (%) is preserved.
+ <code>https://example.org/foo bar</code> → <code>https://example.org/foo%20bar</code>;
+ <code>https://example.com/%</code> parses distinctly from <code>https://example.com/%25</code>.
+</ul>
+
+
+<h3 id=uri-iri-differences-model class=no-num>Different model</h3>

I would lead with this section and make it stronger / change the framing. Something like:

> This standard is based around a parser and serializer, with a structured data type (the URL record) as the intermediary. RFC 3986 and RFC 3987 are based around ABNF grammars, which can be run against input strings to produce a boolean "matches / doesn't match" verdict. (In some cases, one can infer a decomposition of the input strings into substrings by looking at which sub-productions of the RFCs match which substrings of the input. But this is tricky to do consistently.)

Similarly, it may be worth mentioning how the URL Standard approaches normalization and relative URL resolution as something done during parsing, instead of having separate [Reference Resolution](https://datatracker.ietf.org/doc/html/rfc3986#section-5) and [Normalization and Comparison](https://datatracker.ietf.org/doc/html/rfc3986#section-6) steps (especially the complex multi-step comparison ladder). Although that touches on https://github.com/whatwg/url/issues/729 and related issues.

> +<ul>
+ <li><p>Non-ASCII input is percent-encoded rather than rejected.
+ <code>https://example.com/café</code> → <code>https://example.com/caf%C3%A9</code>.
+
+ <li><p><a>Special schemes</a> treat U+005C (\) as U+002F (/),
+ tolerate any number of slashes after the scheme, and mandate an authority (though for
+ <code>file:</code> the host may be empty).
+ <code>http:\\example.com\foo</code> → <code>http://example.com/foo</code>;
+ <code>http:example.com/</code> → <code>http://example.com/</code>.
+
+ <li><p>Because non-<code>file</code> special schemes reject an empty host, extra slashes can
+ consume the input meant to be the path. <code>http:///path</code> → <code>http://path/</code>.
+ By contrast, <code>file:///path</code> parses as expected (empty host, path
+ "<code>/path</code>").
+
+ <li><p>Domain hosts of special URLs go through UTS #46 ToASCII, which converts Unicode labels

This bullet is inaccurate ("Non-special hosts are preserved verbatim") and less informative than it could be. Maybe something like

> Non-ASCII domain host treatment differs. Special URLs go through UTS \#\46's ToASCII, which converts Unicode labels to Punycode and lowercases ASCII labels: `https://☕.EXAMPLE/` → `https://xn--53h.example/`. Non-special URLs are percent-encoded: `foo://☕.EXAMPLE/` → `foo://%E2%98%95.EXAMPLE/`. This contrasts with 3986, which rejects non-ASCII hosts and only optionally lowercases them, or 3987, which suggests percent-encoding as a default with an older, buggier version to ToASCII from RFC 3490 as a fallback for an undefined set of schemes.

> + <li><p>In an HTML document with a non-UTF-8 legacy encoding, the query of a special URL (other
+ than <code>ws:</code> or <code>wss:</code>) is percent-encoded using that encoding.
+ <code>?smörgåsbord</code> in a <code>windows-1252</code> document → query
+ <code>sm%F6rg%E5sbord</code>.
+
+ <li><p>Leading and trailing <a>C0 control or space</a> are stripped from the input; interior
+ U+0009, U+000A, and U+000D are silently removed.
+ <code>https://exa&#x9;mp&#x0A;le.com/</code> (with a U+0009 between "<code>exa</code>" and
+ "<code>mp</code>" and a U+000A between "<code>mp</code>" and "<code>le.com/</code>") →
+ <code>https://example.com/</code>.
+
+ <li><p>Out-of-grammar code points in the path, query, or fragment are percent-encoded rather
+ than rejected, and a bare U+0025 (%) is preserved.
+ <code>https://example.org/foo bar</code> → <code>https://example.org/foo%20bar</code>;
+ <code>https://example.com/%</code> parses distinctly from <code>https://example.com/%25</code>.
+</ul>

Notable missing cases probably worth including:

- Port normalization (especially default-port elision and restriction to < 2^16)
- Host percent-*decoding*, e.g. parsing of `https://exam%70le.org/` and `https://exa%23mple.org/`.
- IPv6 normalization (similar to what is already mentioned for IPv4)
- Userinfo format; in particular the URL Standard splits at first `:` into username and password but the RFC deprecates the `username:password` format
- Allowing `@` in userinfo and percent-encoding it (i.e. `https://a@b@example.com/`)
- URL code points are a superset of 3987, e.g. `U+FFF0` is allowed by the URL Standard
- Scheme lowercasing

> +
+<p>This appendix is non-normative. It highlights how this standard differs from RFC 3986 and
+RFC 3987, and assumes familiarity with all three. [[RFC3986]] [[RFC3987]]
+
+<p>Most of these differences also produce a <a>validation error</a>, though in practice those are
+seldom reported.
+<!-- file://localhost/ and percent-encoded dot segments are exceptions at least. -->
+
+
+<h3 id=uri-iri-differences-parsers class=no-num>Different parsers</h3>
+
+<p>Under this standard, the following inputs parse differently than under RFC 3986 or RFC 3987:
+
+<ul>
+ <li><p>Non-ASCII input is percent-encoded rather than rejected.
+ <code>https://example.com/café</code> → <code>https://example.com/caf%C3%A9</code>.

This is confusing/wrong as written, since 3986 rejects them but 3987 accepts them as IRIS and then tells you how to convert them to URIs.

> @@ -4205,6 +4205,117 @@ examples of proper naming. [[HTML]]
 
 
 
+<h2 id=uri-iri-differences class=no-num>Differences from RFC 3986 and RFC 3987</h2>
+
+<p>This appendix is non-normative. It highlights how this standard differs from RFC 3986 and
+RFC 3987, and assumes familiarity with all three. [[RFC3986]] [[RFC3987]]
+
+<p>Most of these differences also produce a <a>validation error</a>, though in practice those are

"Most" might be overclaiming, e.g. points 1, 4, 12, 15 do not, plus the ones source-commented already. "Some" or "many" seems better.

> + to Punycode and lowercases ASCII labels. <code>https://☕.EXAMPLE/</code> →
+ <code>https://xn--53h.example/</code>. Non-special hosts are preserved verbatim.
+
+ <li><p>IPv4 addresses accept octal, hexadecimal, and fewer-than-four parts, and are always
+ serialized as dotted-decimal. <code>http://0x7f.1</code> → <code>http://127.0.0.1/</code>;
+ <code>http://1.2.3</code> → <code>http://1.2.0.3/</code>.
+
+ <li><p>Whether a host is IPv4 or a domain is decided by whether it ends in a numeric-looking
+ label. <code>http://255.example</code> parses as a domain; <code>http://example.255</code>
+ fails.
+
+ <li><p>The <code>IPvFuture</code> production is not recognized.
+ <code>http://[v1.fe80]/</code> fails.
+
+ <li><p>RFC 6874 IPv6 zone identifiers are not recognized.
+ <code>http://[fe80::a%25eth0]/</code> fails.

3986 and 3987 also prohibit these, so this is a bit out of place.

> + <li><p>IPv4 addresses accept octal, hexadecimal, and fewer-than-four parts, and are always
+ serialized as dotted-decimal. <code>http://0x7f.1</code> → <code>http://127.0.0.1/</code>;
+ <code>http://1.2.3</code> → <code>http://1.2.0.3/</code>.
+
+ <li><p>Whether a host is IPv4 or a domain is decided by whether it ends in a numeric-looking
+ label. <code>http://255.example</code> parses as a domain; <code>http://example.255</code>
+ fails.
+
+ <li><p>The <code>IPvFuture</code> production is not recognized.
+ <code>http://[v1.fe80]/</code> fails.
+
+ <li><p>RFC 6874 IPv6 zone identifiers are not recognized.
+ <code>http://[fe80::a%25eth0]/</code> fails.
+
+ <li><p>A <code>file:</code> URL with host "<code>localhost</code>" normalizes to an empty host.
+ <code>file://localhost/</code> → <code>file:///</code>.

3986 / 3987's optional scheme normalization also does this; might be worth mentioning similar to other cases.

> +
+ <li><p>RFC 6874 IPv6 zone identifiers are not recognized.
+ <code>http://[fe80::a%25eth0]/</code> fails.
+
+ <li><p>A <code>file:</code> URL with host "<code>localhost</code>" normalizes to an empty host.
+ <code>file://localhost/</code> → <code>file:///</code>.
+
+ <li><p><code>file:</code> URLs recognize Windows drive letters (normalizing U+007C (|) to
+ U+003A (:)) and preserve them across <code>..</code>. <code>file:///C|/demo</code> →
+ <code>file:///C:/demo</code>; <code>..</code> against <code>file:///C:/demo</code> →
+ <code>file:///C:/</code>.
+
+ <li><p>Percent-encoded dot segments always collapse as "<code>.</code>"/"<code>..</code>".
+ <code>/a/%2E%2E/b</code> against <code>http://example.com/</code> →
+ <code>http://example.com/b</code>. RFC 3986 only reaches the same result with optional
+ percent-encoding normalization.

```suggestion
 percent-encoding normalization plus optional dot-segment normalization.
```

> + <li><p>Out-of-grammar code points in the path, query, or fragment are percent-encoded rather
+ than rejected, and a bare U+0025 (%) is preserved.
+ <code>https://example.org/foo bar</code> → <code>https://example.org/foo%20bar</code>;
+ <code>https://example.com/%</code> parses distinctly from <code>https://example.com/%25</code>.
+</ul>
+
+
+<h3 id=uri-iri-differences-model class=no-num>Different model</h3>
+
+<p>This standard can produce a <a for=/>URL</a> whose serialization is not parseable as a URI or
+IRI. The parser (and API) can leave several ASCII code points in components where RFC 3986 would
+reject them.
+
+<p>Depending on the component, a bare U+0025 (%), U+0022 ("), U+005B ([), U+005C (\), U+005D (]),
+U+005E (^), U+0060 (`), U+007B ({), U+007C (|), or U+007D (}) can survive parsing, and
+U+0023 (#) can survive inside a fragment. Any URL that carries such a code point serializes to a

This sentence sounds exhaustive but isn't, e.g. it omits `<`, `>`, `"`, U+0020 in opaque paths.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/url/pull/923#pullrequestreview-4613710575
You are receiving this because you are subscribed to this thread.

Message ID: <whatwg/url/pull/923/review/4613710575@github.com>