Re: What changes to Web Messaging spec are proposed? [Was: Re: Using ArrayBuffer as payload for binary data to/from Web Workers]

On Thu, Jun 2, 2011 at 10:17 PM, Jonas Sicking <jonas@sicking.cc> wrote:

> On Thu, Jun 2, 2011 at 4:41 PM, David Levin <levin@chromium.org> wrote:
> >
> >
> > On Thu, Jun 2, 2011 at 4:24 PM, Jonas Sicking <jonas@sicking.cc> wrote:
> >>
> >> On Thu, Jun 2, 2011 at 2:01 PM, David Levin <levin@chromium.org> wrote:
> >> >
> >> >
> >> > On Thu, Jun 2, 2011 at 1:27 PM, Glenn Maynard <glenn@zewt.org> wrote:
> >> >> port.postMessage({frameBuffer: frame}, {transfer: [frame], ports:
> >> >> [port]});
> >>
> >> There are two properties of this approach that I like:
> >>
> >> 1. It means that objects which you'd like to transfer ownership are
> >> not "second class citizens" and can live as part of the normal object
> >> graph that is posted, together with metadata that goes with it (or
> >> even as metadata for other things).
> >>
> >> 2. The receiving side doesn't need to worry about the difference, all
> >> it gets is the graph of objects that was sent to it.
> >
> > Yep, I totally agree with this. All of the current solutions being
> discussed
> > satisfy both of these in fact.
> >
> >>
> >> > It also raises questions when I see it. When I list an object there
> does
> >> > it
> >> > imply that all children are also transfered or do I have to list each
> of
> >> > them explicitly as well?)
> >>
> >> None of the objects which allow transferring of ownership has children
> >> so this doesn't appear to be a problem at this time. If it indeed does
> >> turn into a problem, it would seem like a problem no matter what
> >> solution is used, no?
> >
> > Not if all objects are transferred.
>
> Define "all objects". Consider something like:
>
> a = { x: myArrayBuffer1, y: myArrayBuffer2 };
> worker.postMessage(a, { transfer: true });
>
> In this case the 'a' object is obviously not transferred. Or are you
> proposing that it'd be transferred too somehow?
>
> >> > Then I wonder what is the use case for this complexity.
> >> > Why not something simpler like this?
> >> >     port.postMessage({frameBuffer: frame}, {transfer: true, ports:
> >> > [port]});
> >> > where you can just say indicate that you want the message transfered.
> >>
> >> This means that you have to choose between transferring all arrays and
> >> transferring none of them.
> >
> > Yep, why add the complication of picking individual items to transfer
> > over?  (I'm wary of second system syndrome, which I've seen happen many
> > times in various designs.)
> >
> >>
> >> It also makes it much less explicit which
> >> objects ends up being mutated.
> >
> > It is all of them.
>
> Sure, but what "all" includes might not be obvious to the web
> developer in all cases. For example if you're receiving an object from
> some subsystem that you intend to do processing on. You later decide
> to do the processing in a thread and throw that object into an array
> alongside with some other data. Some of the "other data" you are
> throwing in includes some big arrays that you want to avoid copying
> and so you choose to use the transfer rather than copy mode.
>
> Now you all of a sudden start modifying the data that you got from the
> other subsystem. This is data that you might normally not be even
> looking at. The fact that it happened to contain some ArrayBuffers was
> just a side effect of that that was the data structures that the other
> subsystem uses and was easy for it to return to you.
>
> > Here's a simple use case, suppose I create an array of arrays (a 2d
> array)
> > which contains ArrayBuffers.Now I want to transfer this as fast as
> possible
> > using postMessage.
> > What does my code look like for each of these apis?
>
> Your proposal:
> w.postMessage(my2darray, {transfer: true});
> vs.
> w.postMessage(my2darray, Array.concat.apply(Array, my2darray));
>

Hmm, if my2darray is large, the latter is pretty wasteful.


>
> Now show me the code needed to send a message which contains one big
> buffer from you that you want to transfer, along with some data that
> you got from some other piece of code and which you do not want to
> modify and which may or may not contain ArrayBuffers.
>

I think this can be reversed: what if you got some data from some other
piece of code that you do not really control and you want to transfer (i.e.
transfer all the ArrayBuffers that are inside that data)?

As in, the library gives you two methods, compute() that returns data and
present(data) that presents it, and you want to pass data from worker to
main thread without really caring what is inside that data.


> / Jonas
>
>
Thanks!
Dmitry