Re: 2 APIs and the charter

Summary: let's use the term 'front-end' for the thing with the UI, and 
'back-end' for the thing with the storage and static IP address.   Maybe 
we should call the "Social API" the "front-end protocol (or API)" and 
the "Federation Protocol" the "back-end protocol (or API)".

more details inline, including and idea for three major back-end user 
stories....

On 02/11/2015 09:46 PM, henry.story@bblfish.net wrote:
>> On 12 Feb 2015, at 03:54, Evan Prodromou <evan@e14n.com> wrote:
>>
>> On 2015-02-11 08:53 PM, Sandro Hawke wrote:
>>> I agree with your reading of the charter, etc, but I think it still makes sense to imagine two distinct APIs.
>>> [...]
>>>
>> Sandro,
>>
>> Very well said.
>>
>> I think having only a client-to-server API is an interesting exercise that doesn't scale well. It gets particularly tricky if you support following. Either you have:
>>  • a polling mode where clients repeatedly fetch the feeds of every person the user "follows" and merges them into a single "inbox" feed on the client side, or
>>  • the client has to write every update to every follower's server-side inbox feed each time the user updates.
>> In either case, it gets really unwieldy at scales of dozens of followers, much less the tens of thousands you see on a site like identi.ca or the millions you see on a site like Twitter. Fan-out, retries, and exponential backoff become pretty hard to manage from a smartphone application.
>>
>> On top of that, you need to have a universal authentication system that allows Alice, with an account on serverA, to write data as Alice on serverB (and to read data, if you have any kind of privacy controls). Probably client-side certs, I'd imagine from Henry's perspective, but that's a hard sell.
>>
>> -Evan
> That is an interesting interpretation of the charter, along with those by Erik Wilde, and Sandro Hawke  that preceeded. Here are some comments.
>
>   1) That LDP is mentioned in the « Federation Protocol » does not fit very well with the above reading. I can see how  one can use LDP to develop an API for a « server » Actor to  request to be updates about changes to a resource on another machine, but that is an incredibly specific and unusual application of LDP. It is much more likely that LDP would be used for what the charter calls the  « social API ». So that is just another piece of evidence that the Charter is confusing.
>      Given these inconsistent readings of the charter, and the inconsistencies in the charter, it seems to me that  the charter cannot be brandished as a unambigous   guide  how we must proceed, as it has been recently.

Hopefully Arnaud's email lets us move past this point.

On to more interesting bits...

> 2) Optimizations:
> What is efficient will depend on the type of Social Web Server.
>   a) Personal Server:
>     If it is for example a piece of software running on a machine designed for an individual/family such as the Freedom Box [1], then one server will need to go and fetch the data for all its users in any case. Opening thousand or 10 of thousand connections is totally feasible without memry overload if you use the C select call [2] - which is how the  crawlers were written at AltaVista. The equivalent in Java is the NIO framework, which is used by libraries such as Netty (the core of Jboss). And of course modern functional languages such as Scala with Actor programming frameworks such as  http://akka.io  make it much easier to program these very efficiently. These are now well known to JavaScript programmers as asychronouns frameworks. To sum up: a Freedbom Box should be able to open  10 thousand connections and only use up 4 threads.

An aside: as I recall, moving away from select to epoll (and friends) 
was part of the solution to the c10k problem.   Perhaps this is the time 
to acknowledge neither of us has real experience designing and 
maintaining systems with millions of users. Fortunately, in those 
moments where the consensus process really shines, it allows a Working 
Group to solve problems none of its members is capable of solving 
alone.    (although, sadly, that doesn't happen very often)     If we 
want this to happen, we have to fight hard to avoid the Dunning-Kruger 
Effect (where people who lack true expertise are fundamentally blind to 
that fact).    This is a strong reason to favor detailed implementation 
reports and demonstrations.

>    Notice that here there is of course no reason why the iPhone client should be making those connections, which would of course have it run out of battery quite quickly. But by specifying an API as client/server in the sense of a specification of the communication along the wire -  it would of course be the role of the Freedbom Box to fetch remote resources in the background.
>   b) enterprise server: we can discuss the issues involved here
>
> What the existence of a) indicates is that there should be an API that covers all the user stories, and an optimisation API for larer structures, and plain and simple optimisations. But if this is the case then the user stories is not the place to distinguish the APIs.

Maybe we can characterize this in terms of "front-end" protocols and 
"back-end" protocols?   Devices which are interacting with users speak 
the front-end protocols while talking to back-end systems. Back-end 
systems might speak front-end protocols to each other, when acting as 
some kind of virtual front-end (ie a proxy), but more often speak 
back-end protocols to each other.    For email, IMAP and POP are a 
front-end protocols, while SMTP is more of a back-end protocol (although 
clients still use it for sending, I think).   Of course, certain 
specialized back-ends speak IMAP to each other.

Front-end protocols are often stateless, only work with one user at a 
time, and can only be initiated from the front-end.   That is, the 
front-end is a stateless client at which a user authenticates, and the 
back-end is the server.   Window systems (X11 and NeWS) were infamous 
for not following this model -- for them, the front-end is the server 
--  but I think such exceptions are fairly rare.

So, "Social API" is a front-end protocol, where the front-end is the 
client and the back-end is the server.   "Federation Protocol" is a 
back-end protocol, spoken between back-end system.   In back-end 
protocols, the "client" is (always?) the party which initiated the 
particular operation or interaction.

Maybe that language is helpful.

As for user stories....

My sense is that the user experience is almost entirely driven by the 
front-end protocol, so that's where nearly all the user stories will be.

The back-end protocol comes into play in variations on the front-end 
stories, I think.  Basically, the user experience is the same as in the 
ordinary front-end stories, but some fundamental assumptions are 
different.   Specifically, I can think of three main ones:

1.   What if the involved participants are using different back-end 
systems which do not trust each other and only communicate by using the 
back-end protocol?

2.   What if there is massive fan-out, with messages going from one user 
to millions of users?  For example, Justin Bieber tweets, and 60 million 
inboxes get his text within seconds.  Do this without a massive central 
system.

3.   What if there is massive fan-in, where each user sees 
filtered/aggregated information from millions of users?  For example, 
the "trending topics" on Twitter, or just searching on a hashtag.  
Again, do this without a massive central system.

I think that pretty much covers the user stories for the back-end 
protocol.    My guess is this WG will only address (1), and I'm not sure 
it'll even be able to do that.  I think (2) and (3) are still research 
topics.   I'm kind of hoping to get researchers working on them soon.   
(In interviewing grad students, I've seen some interest, but I had to 
say we're not quite ready yet.)

(There are other kinds of assumptions one could tinker with.   Like 
handling various kinds of failures, attacks, and business models. I 
don't think we're there yet.)

> 3) This does not mean that a notification system would not be very useful too (even for a FreedomBox), as a way of avoiding « client » polling ( where the « client ». can be the FreedomBox ) It is true that a notification system will really only work well when the machine making the request notification is always on - so I can see why one could think of it as a « server » protocol.
>
> 4) For authentication and Web Access Control issues I don’t suppose the group is about to build a new authentication protocol. That seems like it should be the work of a separate group.

Yes, indeed.   (I thought the charter specifically said it was 
out-of-scope, but I see it doesn't.  Anyway, yeah, that seems pretty 
out-of-scope to me, and I believe we've talked about that before and 
everyone seemed to agree.)

> We just have to assume we have it available.

Yes, ...   although I think we'll have to do more than assume.   I think 
we'll have to find at least one auth system that people in the group can 
build on.   It doesn't have to be standardized yet, but without 
something which works, we wont be able to demonstrate implementations, 
and the systems wont be very useful.

We may need to standardize the envelop/interface for this functionality 
so it's clear how it plugs into what we're doing.  And then we leave the 
door open for something good & standard to plug in there.

> By the way X509 Client Certificates work just as well for « servers »  as for « clients ». In fact is is a lot easier to use authentication on the « server » because there the developer has full access to all the features of the TLS stack.

I imagine for now folks in the group will try using some 
versions/variants of OAuth, and OpenId, and WebId-TLS for this.

> So in conclusion:
>
>    I don’t see that we really have a way from the charter to be able to sort user stories into ones that would fit the « social API » and ones that would fit « the federated api ».  Apart from the inconsistencies in the charter it seems that the second API is more of an optimisation API which should not be visible from the user stories.
>
> Also we need to find a way to use "client/server" in a way that does not clash with most internet engineering tradition. What is needed is a word for machines that are always on, and machines that are close to the human actor, and that may easily be off for a while. Then we will find that all our APIs ( if more than one is needed ) are client/server, but that some APIs are for machines that are always on.

Hoping I've made useful suggestions on these matters (while flying 
BOS->SFO),

       -- Sandro

>
>
> Henry
>
> [1] http://en.wikipedia.org/wiki/FreedomBox
>      and a short video by Eben Moglen author of the GNU public licence
>       http://vimeo.com/20034456
> [2] http://linux.die.net/man/2/select
>
> Social Web Architect
> http://bblfish.net/
>
>
>

Received on Thursday, 12 February 2015 18:49:57 UTC