Re: Comments on draft MIDI API

On Tue, Jun 26, 2012 at 5:51 PM, Dominique Hazael-Massieux <dom@w3.org>wrote:

> Le mardi 26 juin 2012 à 17:36 +0300, Jussi Kalliokoski a écrit :
>
> >         * sequence<> shouldn't be used as attributes (but instead
> >         arrays should
> >         be used) in MIDIEvent
> >
> > I'm not sure I understand the difference, I thought sequence will be
> > implemented as an array?
>
> "Sequences are always passed by value" hence "Sequences MUST NOT be used
> as the type of an attribute", cf
> http://dev.w3.org/2006/webapi/WebIDL/#idl-sequence
>

 I see, thanks! I'll change that!


> >         * I'm surprised that the enumerate*/get* methods on MIDIAccess
> >         seem seem
> >         to synchronous; I would expect that these operations would
> >         require
> >         enough time that they shouldn't block the main thread; also,
> >         it doesn't
> >         look like the API allows to deal with plugging/unplugging MIDI
> >         devices
> >
> > I don't see how these operations could possibly be long enough to
> > block the main thread for long enough to warrant for an asynchronous
> > behaviour, it's highly unusual for an operation like this to be even
> > one millisecond long.
>
> OK; so that means at any point in time, the browser (and thus the OS)
> always know which devices are (or can be) plugged?
>

The semantics here may be too strong, maybe SHOULD should be used instead
of MUST, as forcing the browser to poll on the devices when the OS doesn't
provide an API for this is a bad idea. And disconnect and connect are not
applicable to MIDI ports anyway (as in, not direct USB connection).


> > And the API does allow for (un)plugging. [1]
>
> But only for devices that the app already knows about and keeps track of
> — or am I missing something? To be more concrete, how would a Web app
> know that the new MIDI keyboard I just bought just got plugged in?
>

According to the current API, the only way would be polling. I've been
thinking of a way to fix this, perhaps an event to MIDIAccess when any
device is plugged in or out would suffice, but I'm not sure yet.


> >         * Uint8Array is not defined in WebIDL; I guess it comes from
> >         WebGL, but
> >         then that spec should probably be referred
> >
> > The WebGL Typed Arrays spec is already referred to. [2]
>
> Oh, indeed, I had missed that. But both Typed Array and High Resolution
> Time  should appear in the normative references as well.
>

Yes, for some reason respec.js didn't apply them automatically, I'll have
to fix that. Thanks for pointing it out.


> >         * MIDIMessage should probably be a dictionary rather than an
> >         interface
> >
> > Why & how?
>
> Hmm... Actually, I see that MIDImessage appears as part of an attribute,
> so maybe that's not a good idea actually...
>
> (why? because for just collection of properties, dictionaries are a
> better fit; how? by replacing  "[NoInterfaceObject] interface" with
> "dictionary" and removing the "attribute" keyword from the definition)
>

I see.


> >         At a higher level, it seems like this API has a very strong
> >         fingerprinting risk (due to enumeration of devices, esp. as
> >         they
> >         themselves have a fingerprint property).
> >
> > Indeed, hence the security model of asking user permission via
> > getMIDIAccess.
>
> But that seems a bit weak — it's not because I'm ready to share one MIDI
> device with a Web app that I would ready for the Web app to know all the
> devices available on my computer — or is the intent that the list of
> enumerable devices be filtered by the user as part of the permission
> request?
>

There's nothing stopping the UA from providing features like this. Another
option the UA could provide is that you could {white,black}list certain
MIDI devices for certain domains. If I recall correctly, there was a
discussion about this before, and the consensus was that it's outside the
scope of the spec. Especially as the biggest security problem of the API is
the fingerprinting which is a merely a question of "Should this website be
allowed to see your MIDIDevices?", providing fewer devices to the website
hardly makes a difference from that POV, since even one device provides a
good pool of entropy.


> >         Also, I idly wonder if getting input from MIDI devices
> >         shouldn't be done
> >         via getUserMedia rather than through its own API; I'm not
> >         really sure
> >         how that would work, but I thought I would still share the
> >         question.
> >
> > This has been discussed a bit already, and the original security model
> > I proposed used getUserMedia, but the concensus (for now) is that MIDI
> > doesn't make that much sense as streams (by default). Instead there
> > are plans to integrate more with the MediaStreams API in a later
> > version, for example giving all MIDI input devices a .stream
> > attribute, but for now there isn't much gained value. We'll revisit
> > this when the MediaStreams API is more widely adopted and we can
> > evaluate benefits such as P2P MIDI communication, cross-context
> > transfer of MIDI events etc.
>
> I guess there are two aspects: dealing with mediastream (for which I can
> see that they don't bring a lot of values for MIDI devices) is the one
> you're responding to; the other one is having a separate API call
> (getMIDIAccess) for something that at least in theory is part of the
> semantics of getUserMedia. It might be that the first aspect determines
> for sure that the second one should be overruled, but it might be useful
> to bring that up for discussion with the MEdia Capture Task Force?
>

The problem with using getUserMedia is that gUM should always provide a
LocalMediaStream, if it suddenly provided a MIDIAccess object it'd be quite
confusing. And then again, having MIDI devices available only as media
streams would be highly counter-intuitive, and being able to enumerate and
remember the user's devices is a hard requirement for the API to be useful
to studio software etc.

Nevertheless, I'd like to see more communication between the Media Capture
Task Force and the Audio WG. :)


> (I guess one reason might be that getUserMedia should really be
> getUserMediaStreams :)
>

Heh, yes, but that would make the name even less compatible with providing
a MIDIAccess object.

Cheers,
Jussi