W3C home > Mailing lists > Public > public-audio@w3.org > April to June 2016

Minutes, 2016-04-08 Web Audio (Atlanta, Day 2)

From: Chris Lilley <chris@w3.org>
Date: Fri, 8 Apr 2016 15:24:28 -0400
To: public-audio@w3.org
Message-ID: <5708056C.3060900@w3.org>


                      Audio WG f2f2, Atlanta, Dday 2

08 Apr 2016

    See also: [2]IRC log

       [2] http://www.w3.org/2016/04/08-audio-irc


           rtoyg, hongchan, BillHofmann, jdsmith, ChrisL_, padenot,
           mdjp, kawai

           cwilso, joe


           BillHofmann, ChrisL


      * [3]Topics
          1. [4]Recharter
          2. [5]Output devices
          3. [6]Output Device Selection
          4. [7]wrap-up and next steps
      * [8]Summary of Action Items
      * [9]Summary of Resolutions

    <BillHofmann> Chair mdjp


    <BillHofmann> scribe BillHofmann

    <BillHofmann> mdjp: should open discussion about whether/how we

    <BillHofmann> mdjp: I believe we should continue, and it's
    collecting new use cases, etc.

    <BillHofmann> mdjp: issue raised at TPAC whether WG should
    split into specific areas

    <BillHofmann> mdjp: also of course relates to Web MIDI.

    <BillHofmann> mdjp: does anyone have objections to rechartering
    with a goal to specing V2, in the same structure as we are in

    <BillHofmann> all: silence :)

    <BillHofmann> mdjp: Joe + Matt need to discuss with Chris
    Lilley re process, but...

    <BillHofmann> mdjp: need to document wide review - WAC, etc are
    good evidence of this

    <BillHofmann> mdjp: testing is also key here. Google/Mozilla -
    any updates on the status?

    <BillHofmann> padenot: would be good to have Joe's intern's

    <BillHofmann> padenot: we have run a few blink tests, and ran
    ok. However, ~95% of mozilla tests are not in Web Platform
    Tests - that'd be good to do

    <BillHofmann> rtoyg_m: good thing is that tests can be run in a
    regular browser

    <BillHofmann> hongchan: advantage vs last year, most tests run
    in OfflineAudioContext

    <BillHofmann> padenot: most tests now run in both Offline and

    <BillHofmann> np.

    <BillHofmann> rtoyg_m: our goal is to cover every user visible
    thing, but a ways away from that. most basic things covered.

    <BillHofmann> jdsmith: asking re coverage required.

    <BillHofmann> mdjp: shows coverage sheet from Joe's intern's

    <BillHofmann> jdsmith: are these tests to be submitted?

    <BillHofmann> mdjp: we need to produce an interoperability

    <BillHofmann> jdsmith: Media goals include common test suite
    run on all browsers; at least two implementations have to pass
    all the tests

    <BillHofmann> mdjp: been going on ever since I joined the WG,
    really need someone to cover

    <BillHofmann> jdsmith: Media WG got help from W3C

    <BillHofmann> mdjp: probably no-one around the table has the
    time to do this

    <BillHofmann> BillHofmann: what is the state of Edge (including
    test coverage)

    <BillHofmann> jdsmith: not sure, took a snapshot of Chromium
    last year.

    <BillHofmann> padenot: there is an independent test suite
    someone created...

    <padenot> [10]https://github.com/mohayonao/web-audio-test-api

      [10] https://github.com/mohayonao/web-audio-test-api

    <BillHofmann> primarily a DOM test - covers all interfaces

    <BillHofmann> padenot: roughly 90% WebAudio coverage on mozilla
    - checking now

    <BillHofmann> mdjp: this is a good framework to fill in from,

    <BillHofmann> mdjp: testing is the main work remaining

    <BillHofmann> jdsmith: it's up to us to decide what interop

    <BillHofmann> mdjp: should discuss with ChrisL - but likely we
    need at least some level of perceptual match for output

    ideally tests decide a pass/fail from script automatically, but
    yes we will need some tests that people have to listen to

    although maybe we can do some clever things with nulling where
    the test passes if there is silence

    taxi saams to be arriving, be there shortly

    <BillHofmann> hongchan: our tests do a lot of functional

    <BillHofmann> padenot: we've been using some techniques where
    you run the signal twice, invert, sum - should be zero

    <BillHofmann> padenot: one version run through graph, other is

    <BillHofmann> rtoyg_m: we do that a lot, as well

    <BillHofmann> rtoyg_m: some require bit accuracy, some times we
    do PSNR tests

    <BillHofmann> rtoyg_m: most everything is completely automatic,
    working to get to 100%

    <BillHofmann> kawai: I know the developer.

    <BillHofmann> mdjp: do we think he might be interested in
    extending the suite to cover functional tests? Can kawai talk
    with him?

    <BillHofmann> BillHofmann: we ought send him a letter rather
    than invite to a call

    <BillHofmann> jdsmith: what would collaboration look like? we
    could contribute cases to this?

    <BillHofmann> mdjp: how does this fit into Web Platform Tests?

    <BillHofmann> padenot: we have a few things in it, not much at
    this point. all new tests get pushed upstream to W3C test suite

    <BillHofmann> kawai: uses testharness.js - if you write to this
    and submit, it's automatically tested.

    <BillHofmann> mdjp: we've got test coverage in two browsers;
    mohayonao tests, web platform tests - so in reasonably good
    state, but need to pull things together


      [11] https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit/LayoutTests/webaudio/

    <hongchan> This is our test suite.

    <hongchan> (Chrome)

    <BillHofmann> jdsmith: we've run these as well, don't know
    status of results, though

    <BillHofmann> padenot: 2-3 of these are run regularly.

    <BillHofmann> mdjp: what do we do for rechartering?

    <BillHofmann> mdjp: new charter should cover new use cases,

    <BillHofmann> ChrisL: BillHofmann, should that cover audio

    <BillHofmann> BillHofmann: yes - though should it be in our WG?

    <BillHofmann> ChrisL: we could call it out as a requirement
    whether or not we do it ourselves

    <BillHofmann> mdjp: How long do we need?

    <BillHofmann> ChrisL: roughly June, for summer holidays.
    includes W3C staff overhead (~2 weeks), balloting, collating

    <BillHofmann> ACTION: ChrisL to draft new charter [recorded in

      [12] http://www.w3.org/2016/04/08-audio-minutes.html#action01]

    <trackbot> Created ACTION-125 - Draft new charter [on Chris
    Lilley - due 2016-04-15].

    <BillHofmann> ACTION: ChrisL: to create implementation report
    [recorded in

      [13] http://www.w3.org/2016/04/08-audio-minutes.html#action02]

    <trackbot> Created ACTION-126 - Create implementation report
    [on Chris Lilley - due 2016-04-15].

    <BillHofmann> ACTION: mdjp to coordinate new use cases
    [recorded in

      [14] http://www.w3.org/2016/04/08-audio-minutes.html#action03]

    <trackbot> Created ACTION-127 - Coordinate new use cases [on
    Matthew Paradis - due 2016-04-15].

    <BillHofmann> mdjp: main outstanding item is testing. we've
    found there's a lot of resources available.

    <BillHofmann> mdjp: We've tried to get community input, just
    hasn't worked. Feels like this just needs someone to

    <BillHofmann> mdjp: is there any scope/help we can get for this
    from W3C (to coordinate getting existing tests into)

    <BillHofmann> ChrisL: Web Platform Test tends to be DOM level

    <BillHofmann> ChrisL: it's a best effort on test coodination

    <BillHofmann> jdsmith: should they be in one location?

    <BillHofmann> rtoyg_m: most of ours are plain HTML

    <BillHofmann> BillHofmann: could they be just pushed into the
    web platform tests?

    <BillHofmann> rtoyg_m: sure

    <BillHofmann> mdjp: what we need, basically, is one (mozilla or
    chrome) to run against.

    <BillHofmann> jdsmith: the Web Platform tests, I believe, are
    WebIDL tests, so might have coverage there

    <BillHofmann> ChrisL: can take a look and see if we have
    someone to help coordinate the test effort.

    <BillHofmann> jdsmith: Media group had someone to assist with

    <BillHofmann> ChrisL: I'll be seeing Philippe on Monday, can
    discuss what he did.

    <BillHofmann> BillHofmann: could the existing Gecko tests be
    pushed up?

    <BillHofmann> padenot: no, different test harness.

    <BillHofmann> mdjp: is there an assumption that we have two
    complete implementations for CR?

    <BillHofmann> padenot: depends on how you count forking, 2-4,
    yes. minimum 2 completely different.

    <BillHofmann> rtoyg_m: many of the Safari tests will fail - way

    <scribe> scribenick: ChrisL

Output devices

Output Device Selection

    jdsmith: discussions at tpac, with media capture and streans,
    but lots of requirements deferred to future version
    ... mostly input devices and streams from webcams etc, notion
    of a single output device, not really thinking about output
    ... want it to be generalized enough to say, video output
    devices, returns none or the ones matching constraints on
    resolution, highDR, etc. needs to be generalized enough
    ... some interest but no timelines. Since then, nothing.
    ... Joe agitated to get things moving, sent in some PR but
    nothing major was changed. There is not even abn output video
    ... commented on joe's proposal got some input from netflicks,
    who care a lot about this
    ... we don't want a proprietary solution

    BillHofmann: they miss a lot of required capabilities

    jdsmith: generally useful for media devices. There is a
    fingerprinting concern. Capture group manage that by requiring
    permission. Seen as a privacy thing.
    ... can persist the permission per-domain, edge does not though

    ChrisL: on the web its notmall to assume there is permission to
    output audio 9we mutter arkly about autoplay videos)

    jdsmith: add a kind for video output

    BillHofmann: not in charter

    mdjp: could be in next charter

    jdsmith: broader than audio, not clear who owns it, media
    taskforce, and incubator to start with
    ... no elegant way to manage the fingerprint concern.

    BillHofmann: can already fingerprint

    ChrisL: fonts are a big fingerprint

    BillHofmann: mostly there is a single default output device.
    maybe query that. example where you plug a media adapter intoa
    tv, yu already gave permission and don't want to see a dialog

    jdsmith: misuse of the api is hard, would need to give a reason
    for acessing devices
    ... need to persist permissions for the domain

    BillHofmann: iphone is very wrmissions oriented, location,
    photos etc. apps deal with by double dialogs.

    jdsmith: can also limit the information that is disclosed.
    constrain to what is needed. permissions that make sense to the

    BillHofmann: people are getting used t the iritating dialogs

    padenot: android has run-time permissions now, used to be

    BillHofmann: idea of constraints is to pick an appropriate
    ... (we wonder about permissions in getUserMedia)

    jdsmith: can't get the label until given permission

    ChrisL: media queries just added a gamut query - sRGB, P3, 2020


      [15] https://drafts.csswg.org/mediaqueries/#color-gamut

    jdsmith: we have a hardware pipeline that is more robust, we
    want to expose that, feed it with different characgteristics,
    let apps choose between them

    BillHofmann: you want codec support too? ecide which source
    stream to select.
    ... could do with an API or with MQ


      [16] https://developer.mozilla.org/en-US/docs/Web/API/Window/matchMedia

    mql = window.matchMedia(mediaQueryString)

    mdjp: multichannel auio devices, want to see if there is better
    than stereo

    BillHofmann: media queries for audio?

    padenot: yes but only FF supports them
    ... can have multile source tags for small or large screens. no
    audio in the MQ

    jdsmith: who would review this, would we need a tag opinion?

    mdjp: constraints means you get a device meeting the
    constraints. not like device enumeration.

    BillHofmann: cant get details

    jdsmith: can enumerate audio devices, but not descrinbed in any

    BillHofmann: so connstraits are channels, sample rate. what is
    the minimal list

    mdjp: output buffer size

    BillHofmann: presumably you get that after permission. then you
    get more detail

    mdjp: mostly this is okay.

    BillHofmann: default likely to be hdmi out

    mdjp: permission side is for a more specialized useage. sound
    card details, etc

    (discussion on switching video output devices, auto switching,

    BillHofmann: do you know sample rate?

    hongchan: yes

    padenot: platforms specifics on the internals of that

    BillHofmann: don't know the color capabilities of the output
    device, gamut, dynamic range, bit depth

    (Bill shows some webrtc option dialogs)


      [17] https://webrtc.github.io/samples/src/content/devices/input-output/

    (we play with this for a bit and try adding new devices, etc)

    BillHofmann: so who does it, is it in rechartering, is it a web
    incubator CG, where does it get done?
    ... we can't wait on getUserMedia, they are specific to the
    webrtc use case only

    mdjp: would audio output be better as a subset group or a
    separate group. its a lot of duplication. needs to tie to
    getUserMedia in some way

    jdsmith: would add output devices
    ... it is broader than audio only, so best not in our charter

    BillHofmann: agreed
    ... so we develop use csases first? and where to send them

    billl (finds mailing list for the audi output spec is media


    BillHofmann: seems like the mediacapture output spec is the
    right one for audio output

    seems semi abandoned, has no video output features.

    BillHofmann: so yes we need something, consrtraints api a good
    target, expose a limited set of features fro audio and video.

    jdsmith: fingerprinting is a bit voodoish, so if we have a use
    case that demands data hwhat is the minimal mitigation we
    ... would like to see incubator work on this, by this summer

    mdjp: separate group with some audio wg involvement.

    BillHofmann: incubator is a good place to refactor this so it
    meets our requirements

    jdsmith: want folks from all the relevant groups
    ... api is fairly thought through, can be implemented and
    evaluated rapidly

    mdjp: can steer the direction so it meets our audio needs

    <scribe> ACTION: joe to liaise with the media capture and
    streams to discuss options for adding audio and video device
    constraint and enumeration, in context of existing group or the
    incubator group [recorded in

      [18] http://www.w3.org/2016/04/08-audio-minutes.html#action04]

    <trackbot> Created ACTION-128 - Liaise with the media capture
    and streams to discuss options for adding audio and video
    device constraint and enumeration, in context of existing group
    or the incubator group [on Joe Berkovitz - due 2016-04-15].

    action ChrisL to figure out who reviews the
    privacy/fingerprinting issue

    <trackbot> Created ACTION-129 - Figure out who reviews the
    privacy/fingerprinting issue [on Chris Lilley - due

    <ChrisL_> cwilso: lets keep the implementation status in the


      [19] https://github.com/WebAudio/web-midi-api/issues/148

    <ChrisL_> cwilso: no way to know the inputs and outputs are on
    the same device. so separate lists, not orered or encapsulated
    by hardware device

    <ChrisL_> ... request is to encapsulate them like that

    <ChrisL_> ... challenge is to create individual hardware
    devices for everything in some cases, depending what the OS
    exposes to us

    <ChrisL_> ChrisL_: if it fails, no worse off then now

    <ChrisL_> cwilso: prefer to not change the midiAccess and
    midiPort except a midi interface object on midiport

    <ChrisL_> ChrisL_: so no backwars compat issue

    <ChrisL_> cwilso: no

    <ChrisL_> mdjp: sounds useful

    <ChrisL_> ChrisL_: +1


      [20] https://github.com/WebAudio/web-midi-api/issues/158

    <ChrisL_> cwilso: we don't say how much you can send at one go.
    large buffers help.

    <ChrisL_> ... if pushing a lot of data, esp on DIN midi, mant
    to see if the buffer is emptying fast enough. ask on remaining
    buffer space

    <ChrisL_> cwilso: right way to do this is writeable streams,

    <ChrisL_> cwilso: tied to ES6 and ES7 features

    <ChrisL_> cwilso: can connect streams together but mostly not a
    huge win. better to return a promise and expose the current
    buffer size

    <ChrisL_> ... domenic worrries it is duplicating streams, but
    only a very small part and we don't need the rest

    <ChrisL_> cwilso: so we need to expose backpressure, to give
    confidence that large transfers don't fail

    <ChrisL_> cwilso: think we should do the minimum viable

    <ChrisL_> cwilso: one area streans does help, can pipe between
    ins and outs.

    <ChrisL_> ChrisL_: but midi thru is not especially hard anyway

    <ChrisL_> cwilso: agreed, except for real time messages

    <ChrisL_> cwilso: moving to promise based made midi a bleeding
    edge thing, has worked well though. worried about streams
    because that is difficult to understand with async and ES7

    <ChrisL_> kawai: agree with having both a send one and the
    stream one. it is useful but not really important, we can work
    around it

    <ChrisL_> cwilso: send does not expose backpressure. but we
    could add it

    <ChrisL_> ... don't think we shold replicate a lot of the
    stream api

    <ChrisL_> BillHofmann: not breaking makes sense, so keep send
    and provide the minimal change

    <ChrisL_> cwilso: ok

    <ChrisL_> BillHofmann: what is the minimum change?

    <ChrisL_> cwilso: add a "send space available" call

    <ChrisL_> ... not a "wait until this much space is available"

    <ChrisL_> cwilso: stream implementations are uncommon as yet

    <ChrisL_> mdjp: what about doing the minimal for v1 and
    re-evaluate for v2?

    <ChrisL_> cwilso: that is one way to do it

    <ChrisL_> ... remembering how windows api works here, is it
    dynamically reallocating

    <ChrisL_> ... duplication of effort

    <ChrisL_> ChrisL_: prefer the minimal backpressure for v1,
    close issue, re-open only if new data shows that is not

    <ChrisL_> cwilso: ok, need to talk to the TAGG and see what
    they think

    <ChrisL_> ... WebUSB and WebBT do not expose backpressure, they
    have an async promise based send model

    <ChrisL_> cwilso: rest of the issues are editorial

    <mdjp> [21]https://github.com/WebAudio/web-audio-api/issues/251

      [21] https://github.com/WebAudio/web-audio-api/issues/251

    <ChrisL_> mdjp: ok, great. now nto the webaudio issue

    <ChrisL_> (cwilso reads)

    <ChrisL_> cwilso: think that joe did a subclassing test which
    failed because of #250. Have to go through create methods o
    audioContext. once fixed, subclassing works as well

    <ChrisL_> ... can we compose an audio node and hook up a bunch
    of other things, an then call connect onto an arbitrary point
    in the graph.

    <ChrisL_> cwilso: can't do constructible audio params

    <ChrisL_> ... think this does work today, crate node and
    override .connect

    <ChrisL_> (cwilso tests)

    <ChrisL_> cwilso: yes, you can override it

    <ChrisL_> ... cant expose as audio params

    <ChrisL_> cwilso: being able to new and audio param and have it

    <ChrisL_> cwilso: closed it because we put it on audio worker

    <ChrisL_> mdjp: so it can stay closed

    <ChrisL_> #134

    <ChrisL_> #367 is postponed to v.2

    <ChrisL_> hongchan: so in terms of subclassing we don't need to
    change anything

    <ChrisL_> cwilso: right, but composition does not, yet

    <ChrisL_> cwilso: audio param is still magic. not newable.
    marshalled across to worker behind the scenes

    <ChrisL_> BillHofmann: would #250 solve that?

    <ChrisL_> padenot: it could. yesterday we talked of making an
    audio param from iside a worklet

    <ChrisL_> Chair: Matt

    <ChrisL_> Meeting: Audio WG f2f2, Atlanta, Day 2

    <ChrisL_> cwilso: or we could say #251 and say 367 back

    <ChrisL_> cwilso: don't care exactly because we need to keep
    going on constructible params, whether it hits v.1 or not

    <ChrisL_> padenot: we should get constructible audionodes
    first, then constructible audio params, look at this issue.

    <ChrisL_> rtoyg: agree

    <ChrisL_> jdsmith: general question - there like 80 issues
    ready for editing, this one is useful but if we wanted to get
    these down to CR in summer, is this where we would be working
    and push to v.next?

    <cwilso> (Sorry, connection froze)

    <ChrisL_> jdsmith: maybe we need a v1-nonblocking flag

    <cwilso> I think this is pretty straightforward to spec; I
    think the question of how hard it is to implement might factor
    in there, but it is one of those fundamental,
    Extensible-Web-Layering type issues.

    <cwilso> hah

    <ChrisL_> BillHofmann: Jerry asked if we should stop adding new
    stuff and deal with the easier v.1 issues, pushing others to
    v.2 or v.1.nonblocking ie desired but at risk

    <ChrisL_> cwilso: it is an extensible web layering issue

    <ChrisL_> .. avoiding magic

    <ChrisL_> ... without this we can't extend

    <ChrisL_> ChrisL_: agree this is architectural

    <ChrisL_> padenot: if we realy want composite nodes, we need
    audio param remapping

    <ChrisL_> cwilso: useful when making a unity node, or a looping
    single sample; other one is four params exposed on a composite

    <ChrisL_> padenot: what about a composite audio param that
    connects to two audio params

    <ChrisL_> cwilso: right, you need to be able to do that

    <ChrisL_> cwilso: title needs to be "constructible and

    <ChrisL_> mdjp: so, are we in v.1 or v.next?

    <ChrisL_> padenot: we keep adding stuff to v.1. Reluctant to
    but this is important

    <ChrisL_> padenot: review backlog next week and triage

    <ChrisL_> rtoyg: fair amount of design work in this one, too

    <ChrisL_> rtoyg: not clear how it is supposed to work

    <ChrisL_> mdjp: put a time tlimit on investigating this. Likely
    a v.1 requirement.

    <ChrisL_> mdjp: lets discuss in two weeks (no call next week)

    <ChrisL_> mdjp: agenda is pretty much cleared now

    <ChrisL_> Chair: Matt

    <ChrisL_> Meeting: udio WG f2f2, Atlanta, Day 2

    <ChrisL_> Meeting: Audio WG f2f2, Atlanta, Day 2

wrap-up and next steps

    <ChrisL_> rtoyg: we have 13 open pull requests

    <ChrisL_> padenot: will look at those next week

    <ChrisL_> [22]https://github.com/WebAudio/web-audio-api/pulls

      [22] https://github.com/WebAudio/web-audio-api/pulls

    <ghaudiobot> [web-audio-api] rtoy closed pull request #770: Fix
    #769 by correcting the formulas

      [23] https://github.com/WebAudio/web-audio-api/pull/770

    <ChrisL_> meeting adjourned

Summary of Action Items

    [NEW] ACTION: ChrisL to draft new charter [recorded in
    [NEW] ACTION: ChrisL: to create implementation report [recorded
    [NEW] ACTION: joe to liaise with the media capture and streams
    to discuss options for adding audio and video device constraint
    and enumeration, in context of existing group or the incubator
    group [recorded in
    [NEW] ACTION: mdjp to coordinate new use cases [recorded in

      [24] http://www.w3.org/2016/04/08-audio-minutes.html#action01
      [25] http://www.w3.org/2016/04/08-audio-minutes.html#action02
      [26] http://www.w3.org/2016/04/08-audio-minutes.html#action04
      [27] http://www.w3.org/2016/04/08-audio-minutes.html#action03

Summary of Resolutions

    [End of minutes]

Chris Lilley
Technical Director, W3C Interaction Domain
Received on Friday, 8 April 2016 19:24:38 UTC

This archive was generated by hypermail 2.3.1 : Friday, 8 April 2016 19:24:38 UTC