Re: Using ArrayBuffer as payload for binary data to/from Web Workers

On Thu, May 26, 2011 at 8:20 PM, Jonas Sicking <jonas@sicking.cc> wrote:

> On Fri, Apr 22, 2011 at 6:26 PM, Kenneth Russell <kbr@google.com> wrote:
> > On Mon, Mar 7, 2011 at 6:17 PM, Kenneth Russell <kbr@google.com> wrote:
> >> On Mon, Mar 7, 2011 at 5:18 PM, Chris Marrin <cmarrin@apple.com> wrote:
> >>>
> >>> On Mar 7, 2011, at 4:46 PM, Kenneth Russell wrote:
> >>>
> >>>> On Mon, Mar 7, 2011 at 3:54 PM, Glenn Maynard <glenn@zewt.org> wrote:
> >>>>> On Mon, Mar 7, 2011 at 6:05 PM, Chris Marrin <cmarrin@apple.com>
> wrote:
> >>>>>>
> >>>>>> Now that ArrayBuffer has made its way into XHR, I think it would be
> >>>>>> reasonable to somehow use this new object type as a way to pass data
> to and
> >>>>>> from Workers without copying. I've seen hints and thoughts about
> this here
> >>>>>> and there, but I've never seen a formal discussion. I'm not even
> sure if
> >>>>>> webapps is the place for this discussion, although it seems like a
> >>>>>> reasonable place. Please let me know if there is a better place.
> >>>>>
> >>>>> ArrayBuffer is the most obvious use for zero-copy messaging, but I
> don't
> >>>>> think it should be limited to it...
> >>>>>
> >>>>>> Has there been discussion anywhere that I've missed?
> >>>>>
> >>>>> Probably not the only one, but check the "WebWorkers and images"
> thread on
> >>>>> whatwg.
> >>>>
> >>>> There's definitely interest among the editors of the Typed Array spec
> >>>> in revising the spec to support zero-copy data transfers to and from
> >>>> web workers. In informal offline discussions, there was a tentative
> >>>> plan to put up a new draft for discussion within the next month or so.
> >>>> A goal was to prototype it before solidifying a spec so that we can be
> >>>> assured it will work well for real-world use cases.
> >>>
> >>> Yeah, I guess the question is whether we should put the functionality
> into ArrayBuffer, or into a wrapper class which would part of the Web Worker
> spec. The latter might make it easier to add other resources (like image and
> canvas) at some point. But I agree, it should be implemented before
> finalizing anything.
> >>>
> >>> Did I hear you volunteer to add a strawman proposal to the Typed Array
> spec? :-)
> >>
> >> Yes, you did. :-)
> >
> > The editors' draft of the typed array spec has been updated with a
> > strawman proposal for this zero-copy, transfer-of-ownership behavior:
> >
> > http://www.khronos.org/registry/typedarray/specs/latest/
> >
> > Feedback would be greatly appreciated. For the purposes of keeping the
> > conversation centralized, it might be helpful if we could use the
> > public_webgl list; see
> > https://www.khronos.org/webgl/public-mailing-list/ .
>
> While I see the need for this, i think it will be very surprising to
> authors that for all other data, postMessage is purely a read-only
> action. However for ArrayBuffers it would not be. There are two ways
> we can improve this situation:
>
> 1. Add a separate method next to postMessage which has the prescribed
> functionality. This also has the advantage that it lets users choose
> if they want the transfer-ownership functionality or not, for example
> for cases when performance isn't as big requirement, and when
> ArrayBuffers are small enough that the transferring ownership logic
> adds more overhead than memory copying logic would.
>
> 2. Add a separate argument to postMessage, similar to the 'ports'
> argument, which contains a list of array buffers whose ownership
> should be transferred.
>

Riffing off idea #2, the second argument could be an array of objects who's
ownership should be transferred. For now only ArrayBuffers would be legal
objects but at some point in the future other types of objects could be
added (not sure what those objects would be but that's a much more flexible
interface than #1. You can chose to copy some ArrayBuffers and transfer
others.


>
> / Jonas
>
>