Re: [whatwg/url] Web compatibility issue with various unknown (external) protocols like ed2k (Issue #815)

I suspect they are inherited from RFC-2396:

> Other characters are excluded because gateways and other transport
> agents are known to sometimes modify such characters, or they are
> used as delimiters.
>
> unwise      = "{" | "}" | "|" | "\\" | "^" | "[" | "]" | "`"
>
> Data corresponding to excluded characters must be escaped in order to
> be properly represented within a URI.

https://www.ietf.org/rfc/rfc2396.txt

I further suspect the latter part ("they are used as delimiters") is much more common than gateways or other transport agents modifying these characters in URLs. I believe this is also the reason why we have a "special query" percent-encode set, which includes single-quotes/apostrophes -- because JS allows using those characters to delimit string literals, if you substitute in a URL which includes an unescaped single-quote, you could unexpectedly terminate the string literal and attackers might use this to inject code.

But really, this whole idea of URL's escaping delimiter characters of popular enclosing document formats is inherently flawed. Consider that parentheses are allowed without escaping, and by some cruel irony are used by the Markdown document format specifically for delimiting URLs.

Rust and Swift source code allow user-customisable delimiters for string literals (e.g. `r#"..."#` in Rust, where the number of `#`s is customisable so you can include unescaped `#` in the body), which is a much better solution.

So yeah, at least for characters where there are not likely to be any web-related delimiter issues (vertical bar, curly braces, etc), I think we can afford to be more relaxed and allow them to be used without escaping.

-- 
Reply to this email directly or view it on GitHub:
https://github.com/whatwg/url/issues/815#issuecomment-2043214344
You are receiving this because you are subscribed to this thread.

Message ID: <whatwg/url/issues/815/2043214344@github.com>