Re: Content-Negotiation Issues (conneg)

I've tried to avoid content negotiation for the same reasons that Ryan
mentioned. The one place in my ActivityPub implementation that needs it
is handling a request for the actor ID. While the spec doesn't require
content negotiation here, in practice I don't think it can be avoided:
that URL is going to be opened in a web browser by Mastodon to view the
user's profile, so the server has to be smart about returning JSON or
HTML.

— Manton

P.S. Hi everyone. I just joined the SWICG this week. Thanks for all your
work moving things forward.

On March 24, 2023, Erin Shepherd <erin.shepherd@e43.eu> wrote:
> As the spec itself says: nothing in ActivityPub strictly requires use
> of content negotiation.
>
> A client may hardcode the Accept header value. The server may ignore
> it and always serve ActivityStreams. These two implementations will
> interoperate.
>
> The spec “merely” grants permission for servers to implement content
> negotiation, and basically states that the requirements of the
> specification (for GET requests at least) only apply when the AS2
> media type is negotiated.
>
> In practice content negotiation has proven useful by
>
>  *  Permitting removal of the need for any indirection (via
>     rel=alternative type links) when a user passes the URI of a post
>     (presumably in a HTML rendered form)
>  *  Permitting in-place migrations (e.g. from OStatus) without needing
>     to query every old post for its’ AS2 ID
>
>
> Content negotiation is one of the trickier, less understood, and often
> troublesome portions of HTTP, but I must say that in practice it has
> proven useful.
>
> It is probably not the best approach to take for “closed world” (i.e.
> one company/one system, non distributed/federated) APIs, but it seems
> in practice to be one of the best approaches for “open world” APIs
>
> > As a mental exercise, contemplate doing the OStatus to AP migration
> > without it and without 1 request per historical post, without
> > dictating the URI structure that implementations use
> >
> > I’m sure there are other ways of doing it, but contemplate
> > alternative options which work n
>
>
>  - Erin
>
>
> > On 24 Mar 2023, at 18:48, Bob Wyman <bob@wyman.us
> > <mailto:bob@wyman.us>> wrote:
> >
> > Elsewhere, we started talking about issues with Content Negotiation
> > as required by ActivityPub 3.2
> > <https://www.w3.org/TR/activitypub/#retrieving-objects>. I'm copying
> > the previous messages here to ensure better retrievability in the
> > future and to prevent distraction from the main subject of the other
> > thread. If further comments are made on this subject, please reply
> > to this message, not the other one.
> >
> > ---------- Forwarded message ---------
> > From: Bob Wyman <bob@wyman.us <mailto:bob@wyman.us>>
> > Date: Fri, Mar 24, 2023 at 12:42 PM
> > Subject: Re: Should the specs be forked and maintained elsewhere?
> >
> > Melvin, Ryan,
> > Could you provide at least a short summary of why "content
> > negotiation" has not lived up to expectations? It is good to
> > know that it is non-optimal. It would be better to know why..
> >
> > bob wyman
> >
> > ---------- Forwarded message ---------
> > From: Melvin Carvalho <melvincarvalho@gmail.com
> > <mailto:melvincarvalho@gmail.com>>
> > Date: Fri, Mar 24, 2023 at 1:14 PM
> >
> > pá 24. 3. 2023 v 17:42 odesílatel Bob Wyman <bob@wyman..us
> > <mailto:bob@wyman.us>> napsal:
> > > Melvin, Ryan,
> > > Could you provide at least a short summary of why "content
> > > negotiation" has not lived up to expectations? It is good to
> > > know that it is non-optimal. It would be better to know why.
> >
> >
> > Hard to configure, hard to debug, hard to make backwards compatible
> > with the existing web.
> >
> > It's not easy to cache globally because each document have different
> > etags.  It can open some security holes for servers not configured
> > properly.
> >
> > Below is a sample of what Dan Brickley (original chair of the social
> > web XG, author of FOAF, runs schema.org <http://schema.org/>) has to
> > say on content negotiation.  You could also repeat the search for
> > conneg.
> >
> > e.g. "the content negotiation aspect of linked data has been
> > massively oversold"
> >
> > <https://twitter.com/search?q=%40danbri%20content%20negotiation&src=typed_query>
> >
> > Id rather it was optional that required. 
> >
> > ---------- Forwarded message ---------
> > From: Ryan Barrett <public@ryanb.org <mailto:public@ryanb.org>>
> > Date: Fri, Mar 24, 2023 at 1:16 PM
> >
> > Sure! And feel free to redirect to a different venue if we're
> > getting off topic. Off the top of my head:
> > 1. Most web developers are not aware that conneg exists. This is
> >    usually fine, since it's uncommon, but can be an unpleasant
> >    surprise when they first hit it and don't understand it.
> > 2. The `Accept` header is notoriously complicated to parse,
> >    generate, and fully comply with. Imo much of that complexity is
> >    unneeded in practice.
> > 3. It often breaks caching when it's first introduced. Internal and
> >    external caches generally ignore Content-Type and Accept (etc) by
> >    default, so when conneg is first introduced, clients often get
> >    the wrong response type. Developers learn the hard way that they
> >    need to add Accept to Vary and to their framework's cache.
> > 4. URLs leak across contexts. They may start out within a single
> >    service or API, but they gradually proliferate into user-visible
> >    links, object identifiers in other systems, add-on services,
> >    third party tools, etc. These generally don't know the original
> >    conneg requirements, which causes unpleasant surprises.
> >
> > I think most of this boils down to: modality generally considered
> > harmful. When something always behaves the same way, it's reliable
> > and easy to use. When it behaves differently based on something far
> > away in the environment that you may not know exists, it's
> > unreliable and surprising. Add in a very large ecosystem of
> > independent tools that all need to interoperate, often in fine-
> > grained ways, and you have a recipe for failure.
> >
> > ---------- Forwarded message ---------
> > From: Aaron Gray <aaronngray@gmail.com
> > <mailto:aaronngray@gmail.com>>
> > Date: Fri, Mar 24, 2023 at 1:22 PM
> >
> > On Fri, 24 Mar 2023 at 16:49, Aaron Gray <aaronngray@gmail.com
> > <mailto:aaronngray@gmail.com>> wrote:
> > > The standards are there for a reason we should be following them.
> > > Providing access to optional extra typed fields on a post is not
> > > complex.
> >
> >
> > To clarify I was referring to JSON-LD and @Content, not conneg HTTP
> > Content-Type, Accept content negotiation.
> >
> > Regards,
> >
> > Aaron
> >
> > ---------- Forwarded message ---------
> > From: Benjamin Goering <ben@bengo.co <mailto:ben@bengo.co>>
> > Date: Fri, Mar 24, 2023 at 1:23 PM
> >
> > http content negotiation is normatively required by the activitypub
> > spec (3.2).
> > I'll also point out that some URIs from the TRs already support it.
> >
> > ```
> > > bengo@bengo ~ ⚡  curl -H "Accept:
> > > application/json" <https://www.w3.org/ns/activitystreams> | head -
> > > n 10
> > >  % Total % Received % Xferd Average Speed Time Time Time Current
> > >  Dload Upload Total Spent Left Speed
> > > 100 7985 100 7985 0 0 39142 0 --:--:-- --:--:-- --:--:-- 40125
> > > {
> > >   "@context": {
> > >     "@vocab": "_:",
> > >     "xsd": "<http://www.w3.org/2001/XMLSchema#>",
> > >     "as": "<https://www.w3.org/ns/activitystreams#>",
> > >     "ldp": "<http://www.w3.org/ns/ldp#>",
> > >     "vcard": "<http://www.w3.org/2006/vcard/ns#>",
> > >     "id": "@id",
> > >     "type": "@type",
> > >     "Accept": "as:Accept",
> > > bengo@bengo ~ ⚡  curl -H "Accept:
> > > text/html" <https://www.w3.org/ns/activitystreams> | head -n 10
> > >  % Total % Received % Xferd Average Speed Time Time Time Current
> > >  Dload Upload Total Spent Left Speed
> > >  0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0<!DOCTYPE html><html
> > > dir="ltr" typeof="bibo:Document " prefix="bibo:
> > > <http://purl.org/ontology/bibo/> w3p: <http://www.w3.org/2001/02pd/rec54#> as: <https://www.w3.org/ns/activitystreams#>"
> > > lang="en"><head><meta charset="utf-8"><meta name="generator"
> > > content="ReSpec 20.7.1"><meta name="viewport"
> > > content="width=device-width, initial-scale=1, shrink-to-
> > > fit=no"><meta property="dc:language" content="en" lang="">
> > >     <title>ActivityStreams 2.0 Terms</title>
> > >
> > >
> > >
> > >  
> > > <style>/*****************************************************************
> > >  * ReSpec 3 CSS
> > >  * Robin Berjon - <http://berjon.com/>
> > >  *****************************************************************/
> > >
> > > 100 33068 0 33068 0 0 148k 0 --:--:-- --:--:-- --:--:-- 153k
> > > curl: (23) Failure writing output to destination
> >
> > ```
> > --------- Forwarded message ---------
> > From: Melvin Carvalho <melvincarvalho@gmail.com
> > <mailto:melvincarvalho@gmail.com>>
> > Date: Fri, Mar 24, 2023 at 1:35 PM
> > Subject: Re: Should the specs be forked and maintained elsewhere?
> >
> > pá 24. 3. 2023 v 18:23 odesílatel Benjamin Goering <ben@bengo.co
> > <mailto:ben@bengo.co>> napsal:
> > > http content negotiation is normatively required by the
> > > activitypub spec (3.2).
> > > I'll also point out that some URIs from the TRs already support
> > > it.
> >
> >
> > How do you ensure that each accept header gives back the same
> > triple?
> >
> > For example, do they in this case.
> >
> > Not a pedantic point, I've been trying to fix this for years in
> > Solid, and its made harder by the fact that the developers dont know
> > how it works.  In fact the maintainer of the reference
> > implementation said to me recently: "What's content negotiation". 
> > It's hard to fix these things.
> >  
> >
> > ---------- Forwarded message ---------
> > From: Melvin Carvalho <melvincarvalho@gmail.com
> > <mailto:melvincarvalho@gmail.com>>
> > Date: Fri, Mar 24, 2023 at 1:41 PM
> >
> > pá 24. 3. 2023 v 18:34 odesílatel Melvin Carvalho
> > <melvincarvalho@gmail.com <mailto:melvincarvalho@gmail.com>> napsal:
> > >
> > >
> > > pá 24. 3. 2023 v 18:23 odesílatel Benjamin Goering <ben@bengo.co
> > > <mailto:ben@bengo.co>> napsal:
> > > > http content negotiation is normatively required by the
> > > > activitypub spec (3.2).
> > > > I'll also point out that some URIs from the TRs already support
> > > > it.
> > >
> > >
> > > How do you ensure that each accept header gives back the same
> > > triple?
> > >
> > > For example, do they in this case.
> > >
> > > Not a pedantic point, I've been trying to fix this for years in
> > > Solid, and its made harder by the fact that the developers dont
> > > know how it works.  In fact the maintainer of the reference
> > > implementation said to me recently: "What's content negotiation". 
> > > It's hard to fix these things.
> >
> >
> > Let me add an extra point
> >
> > Pretty much all the conneg ive seen is broken (e.g. on mastodon too)
> >
> > It's generally broken on vocabs ive seen too.  But nobody actually
> > cares.  Because, people dont actually use vocabs.  They may in
> > future e.g. for xsd:typing but they are barely noticeable today. 
> > The whole overhead of the thing I would argue provides very little
> > developer value other than a documentation where they can copy and
> > paste.
> >  
> >
>
>