Re: Melvin's microfed.org proposal. Is there a better way forward?

Erin wrote:

> ActivityPub is not a polled protocol; I don't know where you get this
> idea. This is especially the case for AP in practice, i.e. where the
> Mastodon API (or similar) is typically used for client-to-server
> interactions, which has Server Sent Event and Websocket streaming endpoints!


I got this "idea," bizarre as it may sound, by reading the ActivityPub
specification. As defined in that specification, ActivityPub relies on
clients polling servers. An ActivityPub client gets stuff by issuing HTTP
GET requests. That's polling. If a client receives any data from a server
that is not a response to an HTTP GET, then it has implemented some
mechanism that is not defined in the ActivityPub specification. The same
can be said about any ActivityPub server that sends data to a client as
anything other than a response to a GET or POST request -- however it does
it, it ain't ActivityPub. As such, it isn't possible to write a non-polling
client without having implementation-specific knowledge about some
non-standard server's API. The point of interoperability standards is to
eliminate the need to know about specific implementations.

While Mastodon may have implemented "Server-Sent-Events," the use of that
mechanism is part of a private protocol defined by Mastodon, not something
that is defined by ActivityPub. Mastodon does not define ActivityPub.

I suggest that if Mastodon has found it useful to implement Server Sent
Events, that should be a good indication that either that mechanism, or one
that addresses similar issues, such as WebSockets, should be considered as
a candidate for standardization as part of the ActivityPub suite. And, as
Erin suggests elsewhere "it wouldn't be hard to extend" the core specs by
defining optional alternative mechanisms. (Ideally implementations would
always allow fallback to just the standard interfaces). In this community,
there is a great deal of experience with a wide variety of non-polling
interfaces. There are, of course, some interesting problems to deal with.
For instance, unlike WebSockets, ServerSentEvents is
a uni-directional system and the ServerSentEvent specification says that a
browser can only support six concurrent ServerSentEvent connections. These
limitations of ServerSentEvents may be considered acceptable by Mastodon's
implementers, but it isn't clear to me that the properly broader goals of
the SocialWeb community can be reasonably achieved with such limits. In any
case, such issues should be raised and discussed with the goal of defining
a standard approach acceptable to the community at large, not to just one
implementer.

ActivityPub relays are a thing that already exists and are supported by all
> of the major fediverse implementations today.


I am well aware that things called "ActivityPub Relays" exist. However, it
is also apparent that there is no actual ActivityPub specification that
defines what a relay is or how it works. Given that ActivityPub relays have
been in use for quite some time, my guess is that there is sufficient
knowledge to allow one to prepare a specification for them. Often,
standards are used to memorialize and record what has been learned.
Standards don't always define new stuff. So, given that relays exist, and
that they appear to be useful, we should formally define their behaviour.
We should also consider whether the current behavior of relays is actually
sufficient to address anticipated needs.

bob wyman


On Thu, May 4, 2023 at 7:06 PM Erin Shepherd <erin.shepherd@e43.eu> wrote:

>
> On Thu, 4 May 2023, at 21:30, Bob Wyman wrote:
>
> Melvin Cavalho proposes
> <https://lists.w3.org/Archives/Public/public-swicg/2023May/0013.html>:
>
> But we should also consider the whole open social web and one space.
> Working to create bridges will benefit everyone.  Today I started on a
> design of a micro framework for tedi that can implement AP and also other
> protocols, including Solid.
> https://microfed.org/
>
>
> I am concerned that the proposal for yet-another protocol may, in fact,
> take us farther from where we want to go, rather than closer. I can't help
> wondering if there is some way to move forward by taking the best of the
> alternative, existing protocols and defining them as extensions to existing
> SocialWeb protocols or as inter-protocol bridges. For instance, there is
> already "Mostr," a Nostr <-> ActivityPub bridge
> <https://soapbox.pub/blog/mostr-fediverse-nostr-bridge/> that was
> developed to win a bounty posted by fiatjaf
> <https://gist.github.com/fiatjaf/ea7d21e81359e1eb8abcb8805306adaa>.
>
> Mostr acts as an ActivityPub server and as a Nostr client. It has an
> ActivityPub inbox which converts data into Nostr events which it pushes to
> a Nostr relay. It also listens on the relay and federates ActivityPub data
> for relevant Nostr events. The general pattern there seems to be very much
> what one would get from integrating WebSub into the ActivityPub network.
> The conceptual similarities between a WebSub server and a Nostr relay are
> significant. (Except that a Nostr relay sometimes does more than simply
> providing PubSub, store-and-forward, service for messages.)
>
> One of the issues I have with ActivityPub today is that it requires use of
> a specific transport protocol for messages. It does more than simply define
> message content and syntax. Thus, you can't have an ActivityPub-compliant
> interface that uses anything other than HTTP GET and POST. It seems to me
> that this restriction is unnecessary even if most implementers will be
> happy to support only HTTP and aren't concerned about the inherent
> inefficiency or latency that is required by HTTP polling. A more generally
> useful and future-proof ActitvityPub specification would allow
> the implementation of at least client-server APIs over WebSockets. Of
> course, if we were to do that, then clients that spoke "ActivityPub over
> WebSockets" would be well on their way to also speaking Nostr --
> particularly if something like Mostr was available to bridge between the
> application protocols.
>
>
> It's not clear to me what polling you're talking about here. In the C2S
> protocol? Sure, though that's not really in wide use. It's unfortunate
> nothing better is defined there, true, but it wouldn't be hard to extend.
> You're always going to end up with the polling fallback anyway (if nothing
> else but as one way to handle backlog fetches during periods when a client
> is offline)
>
> FWIW you don't *need* to go all the way to WebSockets here; given
> relative proportions of download to upload, Server Sent Events
> <https://html.spec.whatwg.org/multipage/server-sent-events.html#server-sent-events>
> or something similar would be absolutely fine.
>
> But the actual widely deployed bit of ActivityPub is S2S, which is not
> polled.
>
> An ActivityPub server that supported WebSockets wouldn't have to deprecate
> its default HTTP implementation and a server that only supported HTTP
> wouldn't be inherently "deficient." Both interfaces could be easily
> supported, side-by-side. Which interface was used would depend on how a
> client connects to the server. If you connect via WebSockets, you'd be
> subscribed to have updates pushed to you as they are being inserted into
> your InBox. But, if you connected via HTTP, you'd have to poll for updates,
> as you do today. Message formats, sequences, etc. would not have to change.
>
> One of the good things about Nostr relays is that their use solves one of
> the commonly observed problems with ActivityPub as defined. ActivityPub
> messages sent to "Public" are only exchanged between servers that know each
> other. Thus, one's view of message traffic depends heavily on which server
> you use as your primary. However, if we were to define an ActivityPub
> variant of WebSub, which would act much as a Nostr relay does, we'd be able
> to allow servers to send "Public" messages to a WebSub server and for all
> other servers to subscribe to messages from that server. This would reduce
> the cost and complexity of public message distribution as well as the cost
> and complexity of receiving or finding them. Given such intermediary WebSub
> redistributing relays, it would be much easier to maintain a consistent
> view of the global discourse than appears common with ActivityPub today. It
> is also likely that such a system would reduce the current advantage that
> large instances have over small instances. (i.e. The larger your home
> instance, the more complete will be your view of the overall message
> traffic. This large-server advantage is unnecessary.)
>
>
> ActivityPub relays are *a thing that already exists* and are supported by
> *all* of the major fediverse implementations today. If this is a problem
> you (as an instance admin) have, you can subscribe to a relay. There are
> probably hundreds of them, with thousands of instances subscribed! I know
> of ~4 different implementations of AP relay servers as well.
>
> *In practice* many admins of smaller servers, including myself, much
> prefer to not receive the firehose of the entire fediverse; it keeps our
> federated timeline relatively useful, allows us to have longer remote user
> message retention, etc. The fact that we exist in our own bubble is to us a
> feature, not a curse.
>
> Now, yes, AP relays don't replace the direct public message distribution
> process, but that's also sort of the point: on nostr you can't talk to a
> friend who is on a different relay (unless both of your relays are
> connected with each other). You're always going to need direct distribution
> unless both servers can somehow know that they're connected to the same
> relay and trust it's reliability.
>
> One of the nice side-effects of WebSocket use is that it allows us to
> think about protocol extensions that would be odd given ActivityPub's
> reliance on high-latency HTTP polling. For instance, one can imagine
> implementing games like chess over WebSockets, but less so over ActivityPub
> or any other polling protocol. (e.g. Imagine timed moves in a chess game.
> It is really hard to properly time moves if there is polling latency in
> their propagation...) Not surprisingly, there are those who have been
> working on game protocols using Nostr... Given that games are inherently
> "social," I'd like to see similar innovation in the ActivityPub world. Why
> not?
>
> Would anyone else be interested in working on a spec for "*ActivityPub
> over WebSockets*" and also potentially a "*WebSub for
> ActivityPub/ActivityStreams*?" I believe these are things that could be
> developed as optional extensions within the SWICG and without requiring
> modifications to the base protocols.
>
> bob wyman
>
>
> ActivityPub is not a polled protocol; I don't know where you get this
> idea. This is especially the case for AP *in practice*, i.e. where the
> Mastodon API (or similar) is typically used for client-to-server
> interactions, which has Server Sent Event and Websocket streaming
> endpoints! The idea that there is a latency problem in practice is also
> strange to me; the latency between me posting a reply to someone (on a
> different server) and hearing their laptop make a notification bloop is
> not, in my experience, noticably different from the same experience with an
> instant messaging protocol.
>
> It's really not clear to me that WebSockets would actually result in a
> practical or noticable reduction in federation overhead or latency compared
> to just HTTP/2.0 between servers; which is something easy to deploy without
> any protocol changes in AP (and which is probably already deployed; I just
> haven't bothered digging into the HTTP libraries used by the various
> projects).
>
> AP over WebSockets feels to me like a pet project of yours that you've
> talked about a lot based upon an assumption of a need that doesn't seem to
> have a basis in fact. The scaling challenges you - and many people here -
> talk about are not the ones I hear from *the* *actual implementers*.
>
> - Erin
>