Re: ISSUE-124: Futures / Order of parameters (was: Re: Request for JSON-LD API review)

On Wed, Apr 17, 2013 at 7:50 AM, Manu Sporny <msporny@digitalbazaar.com> wrote:
> The JSON-LD API uses node.js-style callbacks. We made this technical
> decision because it was compatible with both server-side and client-side
> code in JavaScript. It was also compatible with asynchronous-style
> programming in Ruby and Python.

Note that node-style callbacks are *also* inconsistent with *every
other callback interface specified in DOM or related specs*.  In
particular, taking a single callback with (error, value) arguments is
not used *anywhere* else.  Consistency arguments don't help you here.

(Note that it doesn't matter *what* callback style you choose -
they're all used by a minority.  The most common is probably a
success/error event pair.  But Node's callback style is, to the best
of the my knowledge, used *nowhere* in W3C specs so far, at least in
specs that matter to the web.)

> It's also important to note that node.js started out with futures as a
> design paradigm and moved away from that design for a number of very
> good reasons:
>
> http://www.futurealoof.com/posts/broken-promises.html

>From what I understand, Node shipped with a crappy model of promises.
Note that there are *at least* four distinct promise models that have
been seriously proposed and used in the web.  Futures is based on the
one that "won" - Promises/A+.  Note that a lot of web-based JS
frameworks *have* adopted promises in the last several years.

Based on the example code in that post, and reading up on the "async"
library, it does seem like the basic problem was that Node's promises
were crappy.  For example, here's the async.map code example:

async.map(arr, fn, finalCB);

It calls fn on every element in the array (with fn able to notify each
invocations completion async).  When everything is done, or something
errors, it calls the finalCB.

Using Futures, it's this:

Future.all.apply(null, arr.map(fn)).then(finalCB);

(Where fn is a function that returns a future.)

This is better than the example code in the referenced blog post, and
importantly, you can use *any* of the future combinators here,
depending on which one you actually need.

(With JS's spread operator, it's even better:
"Future.all(...arr.map(fn)).then(cb)".)

> Tab's post doesn't address the fundamental problems with DOM4 Futures:
>
> 1) The spec isn't done. DOM4 Futures are too experimental for
>    production at this point.

The spec is done.  (You might be referring to the spec's W3C status,
but (a) the spec doesn't live in W3C, and (b) that's more
bureaucracy-arguing, rather than technical arguing.)

> 2) They're not supported in many languages in the same way as DOM4,
>    so anyone wanting to implement the JSON-LD API would have to
>    implement DOM4 Futures.

I don't understand this argument.  JSON-LD has many interesting
web-related requirements, such as HTML fragment parsing.  We intend
browsers to implement Futures anyway, so this won't matter for
browsers.

> 3) Futures aren't as popular yet as callback management. Just taking
>    the two most popular libraries for flow management in node.js
>    (async and q). Callback management (async) has 1,000 libraries that
>    depend on it, futures (q) has around 258.

Chicken/egg problem.  Not a real argument, as long as we think that
the solution is a good direction.

> 4) Mixing callback management with Futures leads to a great deal of
>    pain for developers. It requires each software library to have
>    two flow-control versions. One for callback management and one for
>    futures.

False implication.  You can get away with saying this in Node because
Node has a consistent callback style, so mixing together functions
which use callbacks is actually possible.

In the web, there are at least half a dozen callback strategies used
by major specs.  There is no way to mix them non-painfully.  Adding in
Futures doesn't increase the pain, and as more APIs convert to
Futures, it works to *decrease* it.

> Reason 1: Chaining
> ------------------
>
> You can chain just as easily using the async module:
>
> https://github.com/caolan/async/blob/master/README.md
>
> It works just as well in the browser:
>
> https://github.com/caolan/async/blob/master/README.md#in-the-browser
>
> Reason 2: More Chaining (Deep nesting)
> --------------------------------------
>
> Async, and many of the other callback management libraries, prevent deep
> nesting:
>
> https://github.com/caolan/async/blob/master/README.md#quick-examples

Citing async doesn't help anyone, unless you're only using it to chain
JSON-LD calls, because of the aforementioned half-dozen distinct
callback patterns used for this kind of thing.  Any reasonably-built
consistent way to handle callbacks can have good support for
chaining/etc.  Node-style callbacks with a library like "async" works
fine, as does Futures.  I think we'll end up with more Futures than
more node-style callback APIs.

> Reason 3: Linear callback growth
> --------------------------------
>
> This is annoying to deal with, although there aren't many cases that
> we've found in practice where this leads to deep nesting. I'd say that
> this is a clear win for futures, but it might be solving a problem that
> many developers don't have today.

Yes, it only applies if you're using a bunch of anonymous functions,
and want to apply the same functions for errors/etc in each context.
It's just another point in Futures' favor. ^_^

> Reason 4: Errors are easy to deal with
> --------------------------------------
>
> This is also a solved problem in many of the callback management
> libraries. There is one error function at the end, if any of the
> functions called in a series or in parallel throw an error, all
> subsequent calls are canceled and the error callback is called instead:
>
> https://github.com/caolan/async/blob/master/README.md#seriestasks-callback

Note that this is strictly less powerful than what you can get with
Futures, as you can insert a reject callback into any point in the
chain.  If one of the .then functions doesnt' register a reject
callback, it stays rejected for the next one to pick up.

> Reason 5: Future combinators
> ----------------------------
>
> There are 3 proposed combinators for the DOM4 Futures spec. Async has
> over 10 combinators defined (depending on what your definition for a
> 'combinator' is):
>
> https://github.com/caolan/async/blob/master/README.md#control-flow
>
> 15 combinators exist because the 3 that DOM4 Futures defines are not
> enough to cover the use cases that developers have found in the wild.

Some of these seem to be working around the weirdnesses of callbacks,
and are solved better by first-class representations of the values
like what Futures are.  Others seem quite useful, and I'd be
interested in seeing which ones we could pull into the core!

> Conclusion
> ----------
>
> I remain unconvinced that shifting the current JSON-LD API over to a
> DOM4 Future's approach is the way to go. I do accept that this may be
> the wrong decision in the long term if everyone decides to move over to
> futures. In an attempt to mitigate the risk, perhaps we should also
> define a JSON-LD Futures API in another spec? Maybe we should rename the
> current API to JSON-LD Callback API?
>
> Tab, Boris, did I misunderstand anything above? Did I miss a key concept
> with Futures that are not covered already by async (or many of the other
> callback management libraries)?

The core thing you missed is that there is *no* callback management
library that actually works on the web as it is today.  You can't rely
on Node precedent, because they have a long history of using a single
common callback pattern, making callback management actually
*possible*.  JSON-LD, further, adds a more-or-less brand-new callback
pattern to the set of the half-dozen or so existing patterns, making
the problem worse.  Futures are the attempt to put common callback
management into the web.

At the *very least*, JSON-LD should switch to one of the more common
web callback patterns, such as taking two callbacks in the argument
list, or taking two callbacks in the options argument.  Both of these
are also more common in jQuery APIs.

However, doing so would be a bad idea, since we have an even better
option at our disposal.

~TJ

Received on Wednesday, 17 April 2013 16:54:33 UTC