Re: Content-Negotiation Issues (conneg)

As the spec itself says: nothing in ActivityPub strictly requires use of content negotiation.

A client may hardcode the Accept header value. The server may ignore it and always serve ActivityStreams. These two implementations will interoperate.

The spec “merely” grants permission for servers to implement content negotiation, and basically states that the requirements of the specification (for GET requests at least) only apply when the AS2 media type is negotiated.

In practice content negotiation has proven useful by

Permitting removal of the need for any indirection (via rel=alternative type links) when a user passes the URI of a post (presumably in a HTML rendered form)
(This goes double for cases where rel=alternative is specified by HTML <link> tags, which require some form of HTML parser)
Permitting in-place migrations (e.g. from OStatus) without needing to query every old post for its’ AS2 ID

Content negotiation is one of the trickier, less understood, and often troublesome portions of HTTP, but I must say that in practice it has proven useful.

It is probably not the best approach to take for “closed world” (i.e. one company/one system, non distributed/federated) APIs, but it seems in practice to be one of the best approaches for “open world” APIs

As a mental exercise, contemplate doing the OStatus to AP migration without it and without 1 request per historical post, without dictating the URI structure that implementations use

I’m sure there are other ways of doing it, but contemplate alternative options which work n

 - Erin


> On 24 Mar 2023, at 18:48, Bob Wyman <bob@wyman.us> wrote:
> 
> Elsewhere, we started talking about issues with Content Negotiation as required by ActivityPub 3.2 <https://www.w3.org/TR/activitypub/#retrieving-objects>. I'm copying the previous messages here to ensure better retrievability in the future and to prevent distraction from the main subject of the other thread. If further comments are made on this subject, please reply to this message, not the other one.
> 
> ---------- Forwarded message ---------
> From: Bob Wyman <bob@wyman.us <mailto:bob@wyman.us>>
> Date: Fri, Mar 24, 2023 at 12:42 PM
> Subject: Re: Should the specs be forked and maintained elsewhere?
> 
> Melvin, Ryan,
> Could you provide at least a short summary of why "content negotiation" has not lived up to expectations? It is good to know that it is non-optimal. It would be better to know why.
> 
> bob wyman
> 
> ---------- Forwarded message ---------
> From: Melvin Carvalho <melvincarvalho@gmail.com <mailto:melvincarvalho@gmail.com>>
> Date: Fri, Mar 24, 2023 at 1:14 PM
> 
> pá 24. 3. 2023 v 17:42 odesílatel Bob Wyman <bob@wyman.us <mailto:bob@wyman.us>> napsal:
> Melvin, Ryan,
> Could you provide at least a short summary of why "content negotiation" has not lived up to expectations? It is good to know that it is non-optimal. It would be better to know why.
> 
> Hard to configure, hard to debug, hard to make backwards compatible with the existing web.
> 
> It's not easy to cache globally because each document have different etags.  It can open some security holes for servers not configured properly.
> 
> Below is a sample of what Dan Brickley (original chair of the social web XG, author of FOAF, runs schema.org <http://schema.org/>) has to say on content negotiation.  You could also repeat the search for conneg.
> 
> e.g. "the content negotiation aspect of linked data has been massively oversold"
> 
> https://twitter.com/search?q=%40danbri%20content%20negotiation&src=typed_query <https://twitter.com/search?q=%40danbri%20content%20negotiation&src=typed_query>
> 
> Id rather it was optional that required. 
> 
> ---------- Forwarded message ---------
> From: Ryan Barrett <public@ryanb.org <mailto:public@ryanb.org>>
> Date: Fri, Mar 24, 2023 at 1:16 PM
> 
> Sure! And feel free to redirect to a different venue if we're getting off topic. Off the top of my head:
> Most web developers are not aware that conneg exists. This is usually fine, since it's uncommon, but can be an unpleasant surprise when they first hit it and don't understand it.
> The `Accept` header is notoriously complicated to parse, generate, and fully comply with. Imo much of that complexity is unneeded in practice.
> It often breaks caching when it's first introduced. Internal and external caches generally ignore Content-Type and Accept (etc) by default, so when conneg is first introduced, clients often get the wrong response type. Developers learn the hard way that they need to add Accept to Vary and to their framework's cache.
> URLs leak across contexts. They may start out within a single service or API, but they gradually proliferate into user-visible links, object identifiers in other systems, add-on services, third party tools, etc. These generally don't know the original conneg requirements, which causes unpleasant surprises.
> I think most of this boils down to: modality generally considered harmful. When something always behaves the same way, it's reliable and easy to use. When it behaves differently based on something far away in the environment that you may not know exists, it's unreliable and surprising. Add in a very large ecosystem of independent tools that all need to interoperate, often in fine-grained ways, and you have a recipe for failure.
> 
> ---------- Forwarded message ---------
> From: Aaron Gray <aaronngray@gmail.com <mailto:aaronngray@gmail.com>>
> Date: Fri, Mar 24, 2023 at 1:22 PM
> 
> On Fri, 24 Mar 2023 at 16:49, Aaron Gray <aaronngray@gmail.com <mailto:aaronngray@gmail.com>> wrote:
> The standards are there for a reason we should be following them. Providing access to optional extra typed fields on a post is not complex.
> 
> To clarify I was referring to JSON-LD and @Content, not conneg HTTP Content-Type, Accept content negotiation.
> 
> Regards,
> 
> Aaron
> 
> ---------- Forwarded message ---------
> From: Benjamin Goering <ben@bengo.co <mailto:ben@bengo.co>>
> Date: Fri, Mar 24, 2023 at 1:23 PM
> 
> http content negotiation is normatively required by the activitypub spec (3.2).
> I'll also point out that some URIs from the TRs already support it.
> 
> ```
> bengo@bengo ~ ⚡  curl -H "Accept: application/json" https://www.w3.org/ns/activitystreams <https://www.w3.org/ns/activitystreams> | head -n 10
>   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
>                                  Dload  Upload   Total   Spent    Left  Speed
> 100  7985  100  7985    0     0  39142      0 --:--:-- --:--:-- --:--:-- 40125
> {
>   "@context": {
>     "@vocab": "_:",
>     "xsd": "http://www.w3.org/2001/XMLSchema# <http://www.w3.org/2001/XMLSchema#>",
>     "as": "https://www.w3.org/ns/activitystreams# <https://www.w3.org/ns/activitystreams#>",
>     "ldp": "http://www.w3.org/ns/ldp# <http://www.w3.org/ns/ldp#>",
>     "vcard": "http://www.w3.org/2006/vcard/ns# <http://www.w3.org/2006/vcard/ns#>",
>     "id": "@id",
>     "type": "@type",
>     "Accept": "as:Accept",
> bengo@bengo ~ ⚡  curl -H "Accept: text/html" https://www.w3.org/ns/activitystreams <https://www.w3.org/ns/activitystreams> | head -n 10
>   % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
>                                  Dload  Upload   Total   Spent    Left  Speed
>   0     0    0     0    0     0      0      0 --:--:-- --:--:-- --:--:--     0<!DOCTYPE html><html dir="ltr" typeof="bibo:Document " prefix="bibo: http://purl.org/ontology/bibo/ <http://purl.org/ontology/bibo/> w3p: http://www.w3.org/2001/02pd/rec54# <http://www.w3.org/2001/02pd/rec54#> as: https://www.w3.org/ns/activitystreams# <https://www.w3.org/ns/activitystreams#>" lang="en"><head><meta charset="utf-8"><meta name="generator" content="ReSpec 20.7.1"><meta name="viewport" content="width=device-width, initial-scale=1, shrink-to-fit=no"><meta property="dc:language" content="en" lang="">
>     <title>ActivityStreams 2.0 Terms</title>
> 
> 
> 
>   <style>/*****************************************************************
>  * ReSpec 3 CSS
>  * Robin Berjon - http://berjon.com/ <http://berjon.com/>
>  *****************************************************************/
> 
> 100 33068    0 33068    0     0   148k      0 --:--:-- --:--:-- --:--:--  153k
> curl: (23) Failure writing output to destination
> ```
> --------- Forwarded message ---------
> From: Melvin Carvalho <melvincarvalho@gmail.com <mailto:melvincarvalho@gmail.com>>
> Date: Fri, Mar 24, 2023 at 1:35 PM
> Subject: Re: Should the specs be forked and maintained elsewhere?
> 
> pá 24. 3. 2023 v 18:23 odesílatel Benjamin Goering <ben@bengo.co <mailto:ben@bengo.co>> napsal:
> http content negotiation is normatively required by the activitypub spec (3.2).
> I'll also point out that some URIs from the TRs already support it.
> 
> How do you ensure that each accept header gives back the same triple?
> 
> For example, do they in this case.
> 
> Not a pedantic point, I've been trying to fix this for years in Solid, and its made harder by the fact that the developers dont know how it works.  In fact the maintainer of the reference implementation said to me recently: "What's content negotiation".  It's hard to fix these things.
>  
> 
> ---------- Forwarded message ---------
> From: Melvin Carvalho <melvincarvalho@gmail.com <mailto:melvincarvalho@gmail.com>>
> Date: Fri, Mar 24, 2023 at 1:41 PM
> 
> pá 24. 3. 2023 v 18:34 odesílatel Melvin Carvalho <melvincarvalho@gmail.com <mailto:melvincarvalho@gmail.com>> napsal:
> 
> 
> pá 24. 3. 2023 v 18:23 odesílatel Benjamin Goering <ben@bengo.co <mailto:ben@bengo.co>> napsal:
> http content negotiation is normatively required by the activitypub spec (3.2).
> I'll also point out that some URIs from the TRs already support it.
> 
> How do you ensure that each accept header gives back the same triple?
> 
> For example, do they in this case.
> 
> Not a pedantic point, I've been trying to fix this for years in Solid, and its made harder by the fact that the developers dont know how it works.  In fact the maintainer of the reference implementation said to me recently: "What's content negotiation".  It's hard to fix these things.
> 
> Let me add an extra point
> 
> Pretty much all the conneg ive seen is broken (e.g. on mastodon too)
> 
> It's generally broken on vocabs ive seen too.  But nobody actually cares.  Because, people dont actually use vocabs.  They may in future e.g. for xsd:typing but they are barely noticeable today.  The whole overhead of the thing I would argue provides very little developer value other than a documentation where they can copy and paste.
>  
> 

Received on Friday, 24 March 2023 19:16:36 UTC