Re: Content-Negotiation Issues (conneg)

@erin, your email says
> A client may hardcode the Accept header value. The server may ignore it
and always serve ActivityStreams. These two implementations will
interoperate.

Section 3.2 of activitypub TR says https://www.w3.org/TR/activitypub/
> The client *MUST* specify an Accept header with the application/ld+json;
profile="https://www.w3.org/ns/activitystreams" media type in order to
retrieve the activity.

On Fri, Mar 24, 2023 at 12:40 PM Manton Reece <manton@micro.blog> wrote:

> I've tried to avoid content negotiation for the same reasons that Ryan
> mentioned. The one place in my ActivityPub implementation that needs it is
> handling a request for the actor ID. While the spec doesn't require content
> negotiation here, in practice I don't think it can be avoided: that URL is
> going to be opened in a web browser by Mastodon to view the user's profile,
> so the server has to be smart about returning JSON or HTML..
>
> — Manton
>
> P.S. Hi everyone. I just joined the SWICG this week. Thanks for all your
> work moving things forward.
>
> On March 24, 2023, Erin Shepherd <erin.shepherd@e43.eu> wrote:
>
> As the spec itself says: nothing in ActivityPub strictly *requires* use
> of content negotiation.
>
> A client may hardcode the Accept header value. The server may ignore it
> and always serve ActivityStreams. These two implementations will
> interoperate.
>
> The spec “merely” grants permission for servers to implement content
> negotiation, and basically states that the requirements of the
> specification (for GET requests at least) only apply when the AS2 media
> type is negotiated.
>
> *In practice* content negotiation has proven useful by
>
>
>    - Permitting removal of the need for any indirection (via
>    rel=alternative type links) when a user passes the URI of a post
>    (presumably in a HTML rendered form)
>       - (This goes double for cases where rel=alternative is specified by
>       HTML <link> tags, which require some form of HTML parser)
>    - Permitting in-place migrations (e.g. from OStatus) without needing
>    to query every old post for its’ AS2 ID
>
>
> Content negotiation is one of the trickier, less understood, and often
> troublesome portions of HTTP, but I must say that in practice it has proven
> useful.
>
> It is probably not the best approach to take for “closed world” (i.e. one
> company/one system, non distributed/federated) APIs, but it seems in
> practice to be one of the best approaches for “open world” APIs
>
> As a mental exercise, contemplate doing the OStatus to AP migration
> without it and without 1 request per historical post, without dictating the
> URI structure that implementations use
>
> I’m sure there are other ways of doing it, but contemplate alternative
> options which work n
>
>
>  - Erin
>
>
> On 24 Mar 2023, at 18:48, Bob Wyman <bob@wyman.us> wrote:
>
> Elsewhere, we started talking about issues with Content Negotiation as
> required by ActivityPub 3.2
> <https://www.w3.org/TR/activitypub/#retrieving-objects>. I'm copying the
> previous messages here to ensure better retrievability in the future and to
> prevent distraction from the main subject of the other thread. If further
> comments are made on this subject, please reply to this message, not the
> other one.
>
> ---------- Forwarded message ---------
> From: Bob Wyman <bob@wyman.us>
> Date: Fri, Mar 24, 2023 at 12:42 PM
> Subject: Re: Should the specs be forked and maintained elsewhere?
>
> Melvin, Ryan,
> Could you provide at least a short summary of why "content negotiation"
> has not lived up to expectations? It is good to know that it is
> non-optimal. It would be better to know why.
>
> bob wyman
>
> ---------- Forwarded message ---------
> From: Melvin Carvalho <melvincarvalho@gmail.com>
> Date: Fri, Mar 24, 2023 at 1:14 PM
>
> pá 24. 3. 2023 v 17:42 odesílatel Bob Wyman <bob@wyman.us> napsal:
>
>> Melvin, Ryan,
>> Could you provide at least a short summary of why "content negotiation"
>> has not lived up to expectations? It is good to know that it is
>> non-optimal. It would be better to know why.
>>
>
> Hard to configure, hard to debug, hard to make backwards compatible with
> the existing web.
>
> It's not easy to cache globally because each document have different
> etags.  It can open some security holes for servers not configured properly.
>
> Below is a sample of what Dan Brickley (original chair of the social web
> XG, author of FOAF, runs schema.org) has to say on content negotiation.
> You could also repeat the search for conneg.
>
> e.g. "the content negotiation aspect of linked data has been massively
> oversold"
>
>
> https://twitter.com/search?q=%40danbri%20content%20negotiation&src=typed_query
>
> Id rather it was optional that required.
>
> ---------- Forwarded message ---------
> From: Ryan Barrett <public@ryanb.org>
> Date: Fri, Mar 24, 2023 at 1:16 PM
>
> Sure! And feel free to redirect to a different venue if we're getting off
> topic. Off the top of my head:
>
>    1. Most web developers are not aware that conneg exists. This is
>    usually fine, since it's uncommon, but can be an unpleasant surprise when
>    they first hit it and don't understand it.
>    2. The `Accept` header is notoriously complicated to parse, generate,
>    and fully comply with. Imo much of that complexity is unneeded in practice.
>    3. It often breaks caching when it's first introduced. Internal and
>    external caches generally ignore Content-Type and Accept (etc) by default,
>    so when conneg is first introduced, clients often get the wrong response
>    type. Developers learn the hard way that they need to add Accept to Vary
>    and to their framework's cache.
>    4. URLs leak across contexts. They may start out within a single
>    service or API, but they gradually proliferate into user-visible links,
>    object identifiers in other systems, add-on services, third party tools,
>    etc. These generally don't know the original conneg requirements, which
>    causes unpleasant surprises.
>
> I think most of this boils down to: modality generally considered harmful.
> When something always behaves the same way, it's reliable and easy to use.
> When it behaves differently based on something far away in the environment
> that you may not know exists, it's unreliable and surprising. Add in a very
> large ecosystem of independent tools that all need to interoperate, often
> in fine-grained ways, and you have a recipe for failure.
>
> ---------- Forwarded message ---------
> From: Aaron Gray <aaronngray@gmail.com>
> Date: Fri, Mar 24, 2023 at 1:22 PM
>
> On Fri, 24 Mar 2023 at 16:49, Aaron Gray <aaronngray@gmail.com> wrote:
>
>> The standards are there for a reason we should be following them.
>> Providing access to optional extra typed fields on a post is not complex.
>>
>
> To clarify I was referring to JSON-LD and @Content, not conneg HTTP
> Content-Type, Accept content negotiation.
>
> Regards,
>
> Aaron
>
> ---------- Forwarded message ---------
> From: Benjamin Goering <ben@bengo.co>
> Date: Fri, Mar 24, 2023 at 1:23 PM
>
> http content negotiation is normatively required by the activitypub spec
> (3.2).
> I'll also point out that some URIs from the TRs already support it.
>
> ```
>
>> bengo@bengo ~ ⚡  curl -H "Accept: application/json"
>> https://www.w3.org/ns/activitystreams | head -n 10
>> % Total % Received % Xferd Average Speed Time Time Time Current
>> Dload Upload Total Spent Left Speed
>> 100 7985 100 7985 0 0 39142 0 --:--:-- --:--:-- --:--:-- 40125
>> {
>>   "@context": {
>>     "@vocab": "_:",
>>     "xsd": "http://www.w3.org/2001/XMLSchema#",
>>     "as": "https://www.w3.org/ns/activitystreams#",
>>     "ldp": "http://www.w3.org/ns/ldp#",
>>     "vcard": "http://www.w3.org/2006/vcard/ns#",
>>     "id": "@id",
>>     "type": "@type",
>>     "Accept": "as:Accept",
>> bengo@bengo ~ ⚡  curl -H "Accept: text/html"
>> https://www.w3.org/ns/activitystreams | head -n 10
>> % Total % Received % Xferd Average Speed Time Time Time Current
>> Dload Upload Total Spent Left Speed
>> 0 0 0 0 0 0 0 0 --:--:-- --:--:-- --:--:-- 0<!DOCTYPE html><html
>> dir="ltr" typeof="bibo:Document " prefix="bibo:
>> http://purl.org/ontology/bibo/ w3p: http://www.w3.org/2001/02pd/rec54#
>>  as: https://www.w3.org/ns/activitystreams#" lang="en"><head><meta
>> charset="utf-8"><meta name="generator" content="ReSpec 20.7.1"><meta
>> name="viewport" content="width=device-width, initial-scale=1,
>> shrink-to-fit=no"><meta property="dc:language" content="en" lang="">
>>     <title>ActivityStreams 2.0 Terms</title>
>>
>>
>>
>>
>> <style>/*****************************************************************
>>  * ReSpec 3 CSS
>>  * Robin Berjon - http://berjon.com/
>>  *****************************************************************/
>>
>> 100 33068 0 33068 0 0 148k 0 --:--:-- --:--:-- --:--:-- 153k
>> curl: (23) Failure writing output to destination
>>
> ```
> --------- Forwarded message ---------
> From: Melvin Carvalho <melvincarvalho@gmail.com>
> Date: Fri, Mar 24, 2023 at 1:35 PM
> Subject: Re: Should the specs be forked and maintained elsewhere?
>
> pá 24. 3. 2023 v 18:23 odesílatel Benjamin Goering <ben@bengo.co> napsal:
>
>> http content negotiation is normatively required by the activitypub spec
>> (3.2).
>> I'll also point out that some URIs from the TRs already support it.
>>
>
> How do you ensure that each accept header gives back the same triple?
>
> For example, do they in this case.
>
> Not a pedantic point, I've been trying to fix this for years in Solid, and
> its made harder by the fact that the developers dont know how it works.  In
> fact the maintainer of the reference implementation said to me recently:
> "What's content negotiation".  It's hard to fix these things.
>
>
> ---------- Forwarded message ---------
> From: Melvin Carvalho <melvincarvalho@gmail.com>
> Date: Fri, Mar 24, 2023 at 1:41 PM
>
> pá 24. 3. 2023 v 18:34 odesílatel Melvin Carvalho <
> melvincarvalho@gmail.com> napsal:
>
>>
>>
>> pá 24. 3. 2023 v 18:23 odesílatel Benjamin Goering <ben@bengo.co> napsal:
>>
>>> http content negotiation is normatively required by the activitypub spec
>>> (3.2).
>>> I'll also point out that some URIs from the TRs already support it.
>>>
>>
>> How do you ensure that each accept header gives back the same triple?
>>
>> For example, do they in this case.
>>
>> Not a pedantic point, I've been trying to fix this for years in Solid,
>> and its made harder by the fact that the developers dont know how it
>> works.  In fact the maintainer of the reference implementation said to me
>> recently: "What's content negotiation".  It's hard to fix these things.
>>
>
> Let me add an extra point
>
> Pretty much all the conneg ive seen is broken (e.g. on mastodon too)
>
> It's generally broken on vocabs ive seen too.  But nobody actually cares.
> Because, people dont actually use vocabs.  They may in future e.g. for
> xsd:typing but they are barely noticeable today.  The whole overhead of the
> thing I would argue provides very little developer value other than a
> documentation where they can copy and paste.
>
>
>
>