- From: Chris Lilley <chris@w3.org>
- Date: Fri, 8 Apr 2016 15:24:28 -0400
- To: public-audio@w3.org
https://www.w3.org/2016/04/08-audio-minutes.html
Audio WG f2f2, Atlanta, Dday 2
08 Apr 2016
See also: [2]IRC log
[2] http://www.w3.org/2016/04/08-audio-irc
Attendees
Present
rtoyg, hongchan, BillHofmann, jdsmith, ChrisL_, padenot,
mdjp, kawai
Regrets
cwilso, joe
Chair
Matt
Scribe
BillHofmann, ChrisL
Contents
* [3]Topics
1. [4]Recharter
2. [5]Output devices
3. [6]Output Device Selection
4. [7]wrap-up and next steps
* [8]Summary of Action Items
* [9]Summary of Resolutions
__________________________________________________________
<BillHofmann> Chair mdjp
Recharter
<BillHofmann> scribe BillHofmann
<BillHofmann> mdjp: should open discussion about whether/how we
recharter
<BillHofmann> mdjp: I believe we should continue, and it's
collecting new use cases, etc.
<BillHofmann> mdjp: issue raised at TPAC whether WG should
split into specific areas
<BillHofmann> mdjp: also of course relates to Web MIDI.
<BillHofmann> mdjp: does anyone have objections to rechartering
with a goal to specing V2, in the same structure as we are in
today?
<BillHofmann> all: silence :)
<BillHofmann> mdjp: Joe + Matt need to discuss with Chris
Lilley re process, but...
<BillHofmann> mdjp: need to document wide review - WAC, etc are
good evidence of this
<BillHofmann> mdjp: testing is also key here. Google/Mozilla -
any updates on the status?
<BillHofmann> padenot: would be good to have Joe's intern's
updates
<BillHofmann> padenot: we have run a few blink tests, and ran
ok. However, ~95% of mozilla tests are not in Web Platform
Tests - that'd be good to do
<BillHofmann> rtoyg_m: good thing is that tests can be run in a
regular browser
<BillHofmann> hongchan: advantage vs last year, most tests run
in OfflineAudioContext
<BillHofmann> padenot: most tests now run in both Offline and
regular
<BillHofmann> np.
<BillHofmann> rtoyg_m: our goal is to cover every user visible
thing, but a ways away from that. most basic things covered.
<BillHofmann> jdsmith: asking re coverage required.
<BillHofmann> mdjp: shows coverage sheet from Joe's intern's
test
<BillHofmann> jdsmith: are these tests to be submitted?
<BillHofmann> mdjp: we need to produce an interoperability
report
<BillHofmann> jdsmith: Media goals include common test suite
run on all browsers; at least two implementations have to pass
all the tests
<BillHofmann> mdjp: been going on ever since I joined the WG,
really need someone to cover
<BillHofmann> jdsmith: Media WG got help from W3C
<BillHofmann> mdjp: probably no-one around the table has the
time to do this
<BillHofmann> BillHofmann: what is the state of Edge (including
test coverage)
<BillHofmann> jdsmith: not sure, took a snapshot of Chromium
last year.
<BillHofmann> padenot: there is an independent test suite
someone created...
<padenot> [10]https://github.com/mohayonao/web-audio-test-api
[10] https://github.com/mohayonao/web-audio-test-api
<BillHofmann> primarily a DOM test - covers all interfaces
<BillHofmann> padenot: roughly 90% WebAudio coverage on mozilla
- checking now
<BillHofmann> mdjp: this is a good framework to fill in from,
perhaps
<BillHofmann> mdjp: testing is the main work remaining
<BillHofmann> jdsmith: it's up to us to decide what interop
means
<BillHofmann> mdjp: should discuss with ChrisL - but likely we
need at least some level of perceptual match for output
ideally tests decide a pass/fail from script automatically, but
yes we will need some tests that people have to listen to
although maybe we can do some clever things with nulling where
the test passes if there is silence
taxi saams to be arriving, be there shortly
<BillHofmann> hongchan: our tests do a lot of functional
testing
<BillHofmann> padenot: we've been using some techniques where
you run the signal twice, invert, sum - should be zero
<BillHofmann> padenot: one version run through graph, other is
hand-built
<BillHofmann> rtoyg_m: we do that a lot, as well
<BillHofmann> rtoyg_m: some require bit accuracy, some times we
do PSNR tests
<BillHofmann> rtoyg_m: most everything is completely automatic,
working to get to 100%
<BillHofmann> kawai: I know the developer.
<BillHofmann> mdjp: do we think he might be interested in
extending the suite to cover functional tests? Can kawai talk
with him?
<BillHofmann> BillHofmann: we ought send him a letter rather
than invite to a call
<BillHofmann> jdsmith: what would collaboration look like? we
could contribute cases to this?
<BillHofmann> mdjp: how does this fit into Web Platform Tests?
<BillHofmann> padenot: we have a few things in it, not much at
this point. all new tests get pushed upstream to W3C test suite
<BillHofmann> kawai: uses testharness.js - if you write to this
and submit, it's automatically tested.
<BillHofmann> mdjp: we've got test coverage in two browsers;
mohayonao tests, web platform tests - so in reasonably good
state, but need to pull things together
<hongchan>
[11]https://code.google.com/p/chromium/codesearch#chromium/src/
third_party/WebKit/LayoutTests/webaudio/
[11] https://code.google.com/p/chromium/codesearch#chromium/src/third_party/WebKit/LayoutTests/webaudio/
<hongchan> This is our test suite.
<hongchan> (Chrome)
<BillHofmann> jdsmith: we've run these as well, don't know
status of results, though
<BillHofmann> padenot: 2-3 of these are run regularly.
<BillHofmann> mdjp: what do we do for rechartering?
<BillHofmann> mdjp: new charter should cover new use cases,
etc.
<BillHofmann> ChrisL: BillHofmann, should that cover audio
output?
<BillHofmann> BillHofmann: yes - though should it be in our WG?
<BillHofmann> ChrisL: we could call it out as a requirement
whether or not we do it ourselves
<BillHofmann> mdjp: How long do we need?
<BillHofmann> ChrisL: roughly June, for summer holidays.
includes W3C staff overhead (~2 weeks), balloting, collating
responses
<BillHofmann> ACTION: ChrisL to draft new charter [recorded in
[12]http://www.w3.org/2016/04/08-audio-minutes.html#action01]
[12] http://www.w3.org/2016/04/08-audio-minutes.html#action01]
<trackbot> Created ACTION-125 - Draft new charter [on Chris
Lilley - due 2016-04-15].
<BillHofmann> ACTION: ChrisL: to create implementation report
[recorded in
[13]http://www.w3.org/2016/04/08-audio-minutes.html#action02]
[13] http://www.w3.org/2016/04/08-audio-minutes.html#action02]
<trackbot> Created ACTION-126 - Create implementation report
[on Chris Lilley - due 2016-04-15].
<BillHofmann> ACTION: mdjp to coordinate new use cases
[recorded in
[14]http://www.w3.org/2016/04/08-audio-minutes.html#action03]
[14] http://www.w3.org/2016/04/08-audio-minutes.html#action03]
<trackbot> Created ACTION-127 - Coordinate new use cases [on
Matthew Paradis - due 2016-04-15].
<BillHofmann> mdjp: main outstanding item is testing. we've
found there's a lot of resources available.
<BillHofmann> mdjp: We've tried to get community input, just
hasn't worked. Feels like this just needs someone to
coordinate.
<BillHofmann> mdjp: is there any scope/help we can get for this
from W3C (to coordinate getting existing tests into)
<BillHofmann> ChrisL: Web Platform Test tends to be DOM level
<BillHofmann> ChrisL: it's a best effort on test coodination
<BillHofmann> jdsmith: should they be in one location?
<BillHofmann> rtoyg_m: most of ours are plain HTML
<BillHofmann> BillHofmann: could they be just pushed into the
web platform tests?
<BillHofmann> rtoyg_m: sure
<BillHofmann> mdjp: what we need, basically, is one (mozilla or
chrome) to run against.
<BillHofmann> jdsmith: the Web Platform tests, I believe, are
WebIDL tests, so might have coverage there
<BillHofmann> ChrisL: can take a look and see if we have
someone to help coordinate the test effort.
<BillHofmann> jdsmith: Media group had someone to assist with
this.
<BillHofmann> ChrisL: I'll be seeing Philippe on Monday, can
discuss what he did.
<BillHofmann> BillHofmann: could the existing Gecko tests be
pushed up?
<BillHofmann> padenot: no, different test harness.
<BillHofmann> mdjp: is there an assumption that we have two
complete implementations for CR?
<BillHofmann> padenot: depends on how you count forking, 2-4,
yes. minimum 2 completely different.
<BillHofmann> rtoyg_m: many of the Safari tests will fail - way
behind.
<scribe> scribenick: ChrisL
Output devices
Output Device Selection
jdsmith: discussions at tpac, with media capture and streans,
but lots of requirements deferred to future version
... mostly input devices and streams from webcams etc, notion
of a single output device, not really thinking about output
channels
... want it to be generalized enough to say, video output
devices, returns none or the ones matching constraints on
resolution, highDR, etc. needs to be generalized enough
... some interest but no timelines. Since then, nothing.
... Joe agitated to get things moving, sent in some PR but
nothing major was changed. There is not even abn output video
type
... commented on joe's proposal got some input from netflicks,
who care a lot about this
... we don't want a proprietary solution
BillHofmann: they miss a lot of required capabilities
jdsmith: generally useful for media devices. There is a
fingerprinting concern. Capture group manage that by requiring
permission. Seen as a privacy thing.
... can persist the permission per-domain, edge does not though
ChrisL: on the web its notmall to assume there is permission to
output audio 9we mutter arkly about autoplay videos)
jdsmith: add a kind for video output
BillHofmann: not in charter
mdjp: could be in next charter
jdsmith: broader than audio, not clear who owns it, media
taskforce, and incubator to start with
... no elegant way to manage the fingerprint concern.
BillHofmann: can already fingerprint
ChrisL: fonts are a big fingerprint
BillHofmann: mostly there is a single default output device.
maybe query that. example where you plug a media adapter intoa
tv, yu already gave permission and don't want to see a dialog
box
jdsmith: misuse of the api is hard, would need to give a reason
for acessing devices
... need to persist permissions for the domain
BillHofmann: iphone is very wrmissions oriented, location,
photos etc. apps deal with by double dialogs.
jdsmith: can also limit the information that is disclosed.
constrain to what is needed. permissions that make sense to the
user.
BillHofmann: people are getting used t the iritating dialogs
padenot: android has run-time permissions now, used to be
install-only
BillHofmann: idea of constraints is to pick an appropriate
device
... (we wonder about permissions in getUserMedia)
jdsmith: can't get the label until given permission
ChrisL: media queries just added a gamut query - sRGB, P3, 2020
basically
[15]https://drafts.csswg.org/mediaqueries/#color-gamut
[15] https://drafts.csswg.org/mediaqueries/#color-gamut
jdsmith: we have a hardware pipeline that is more robust, we
want to expose that, feed it with different characgteristics,
let apps choose between them
BillHofmann: you want codec support too? ecide which source
stream to select.
... could do with an API or with MQ
[16]https://developer.mozilla.org/en-US/docs/Web/API/Window/mat
chMedia
[16] https://developer.mozilla.org/en-US/docs/Web/API/Window/matchMedia
mql = window.matchMedia(mediaQueryString)
mdjp: multichannel auio devices, want to see if there is better
than stereo
BillHofmann: media queries for audio?
padenot: yes but only FF supports them
... can have multile source tags for small or large screens. no
audio in the MQ
jdsmith: who would review this, would we need a tag opinion?
mdjp: constraints means you get a device meeting the
constraints. not like device enumeration.
BillHofmann: cant get details
jdsmith: can enumerate audio devices, but not descrinbed in any
way
BillHofmann: so connstraits are channels, sample rate. what is
the minimal list
mdjp: output buffer size
BillHofmann: presumably you get that after permission. then you
get more detail
mdjp: mostly this is okay.
BillHofmann: default likely to be hdmi out
mdjp: permission side is for a more specialized useage. sound
card details, etc
(discussion on switching video output devices, auto switching,
hot-plug)
BillHofmann: do you know sample rate?
hongchan: yes
padenot: platforms specifics on the internals of that
BillHofmann: don't know the color capabilities of the output
device, gamut, dynamic range, bit depth
(Bill shows some webrtc option dialogs)
<BillHofmann>
[17]https://webrtc.github.io/samples/src/content/devices/input-
output/
[17] https://webrtc.github.io/samples/src/content/devices/input-output/
(we play with this for a bit and try adding new devices, etc)
BillHofmann: so who does it, is it in rechartering, is it a web
incubator CG, where does it get done?
... we can't wait on getUserMedia, they are specific to the
webrtc use case only
mdjp: would audio output be better as a subset group or a
separate group. its a lot of duplication. needs to tie to
getUserMedia in some way
jdsmith: would add output devices
... it is broader than audio only, so best not in our charter
BillHofmann: agreed
... so we develop use csases first? and where to send them
billl (finds mailing list for the audi output spec is media
capture)
mediacapture-output
BillHofmann: seems like the mediacapture output spec is the
right one for audio output
seems semi abandoned, has no video output features.
BillHofmann: so yes we need something, consrtraints api a good
target, expose a limited set of features fro audio and video.
jdsmith: fingerprinting is a bit voodoish, so if we have a use
case that demands data hwhat is the minimal mitigation we
propose?
... would like to see incubator work on this, by this summer
mdjp: separate group with some audio wg involvement.
BillHofmann: incubator is a good place to refactor this so it
meets our requirements
jdsmith: want folks from all the relevant groups
... api is fairly thought through, can be implemented and
evaluated rapidly
mdjp: can steer the direction so it meets our audio needs
<scribe> ACTION: joe to liaise with the media capture and
streams to discuss options for adding audio and video device
constraint and enumeration, in context of existing group or the
incubator group [recorded in
[18]http://www.w3.org/2016/04/08-audio-minutes.html#action04]
[18] http://www.w3.org/2016/04/08-audio-minutes.html#action04]
<trackbot> Created ACTION-128 - Liaise with the media capture
and streams to discuss options for adding audio and video
device constraint and enumeration, in context of existing group
or the incubator group [on Joe Berkovitz - due 2016-04-15].
action ChrisL to figure out who reviews the
privacy/fingerprinting issue
<trackbot> Created ACTION-129 - Figure out who reviews the
privacy/fingerprinting issue [on Chris Lilley - due
2016-04-15].
<ChrisL_> cwilso: lets keep the implementation status in the
readme.md
<ChrisL_>
[19]https://github.com/WebAudio/web-midi-api/issues/148
[19] https://github.com/WebAudio/web-midi-api/issues/148
<ChrisL_> cwilso: no way to know the inputs and outputs are on
the same device. so separate lists, not orered or encapsulated
by hardware device
<ChrisL_> ... request is to encapsulate them like that
<ChrisL_> ... challenge is to create individual hardware
devices for everything in some cases, depending what the OS
exposes to us
<ChrisL_> ChrisL_: if it fails, no worse off then now
<ChrisL_> cwilso: prefer to not change the midiAccess and
midiPort except a midi interface object on midiport
<ChrisL_> ChrisL_: so no backwars compat issue
<ChrisL_> cwilso: no
<ChrisL_> mdjp: sounds useful
<ChrisL_> ChrisL_: +1
<ChrisL_>
[20]https://github.com/WebAudio/web-midi-api/issues/158
[20] https://github.com/WebAudio/web-midi-api/issues/158
<ChrisL_> cwilso: we don't say how much you can send at one go.
large buffers help.
<ChrisL_> ... if pushing a lot of data, esp on DIN midi, mant
to see if the buffer is emptying fast enough. ask on remaining
buffer space
<ChrisL_> cwilso: right way to do this is writeable streams,
apparently
<ChrisL_> cwilso: tied to ES6 and ES7 features
<ChrisL_> cwilso: can connect streams together but mostly not a
huge win. better to return a promise and expose the current
buffer size
<ChrisL_> ... domenic worrries it is duplicating streams, but
only a very small part and we don't need the rest
<ChrisL_> cwilso: so we need to expose backpressure, to give
confidence that large transfers don't fail
<ChrisL_> cwilso: think we should do the minimum viable
<ChrisL_> cwilso: one area streans does help, can pipe between
ins and outs.
<ChrisL_> ChrisL_: but midi thru is not especially hard anyway
<ChrisL_> cwilso: agreed, except for real time messages
<ChrisL_> cwilso: moving to promise based made midi a bleeding
edge thing, has worked well though. worried about streams
because that is difficult to understand with async and ES7
features
<ChrisL_> kawai: agree with having both a send one and the
stream one. it is useful but not really important, we can work
around it
<ChrisL_> cwilso: send does not expose backpressure. but we
could add it
<ChrisL_> ... don't think we shold replicate a lot of the
stream api
<ChrisL_> BillHofmann: not breaking makes sense, so keep send
and provide the minimal change
<ChrisL_> cwilso: ok
<ChrisL_> BillHofmann: what is the minimum change?
<ChrisL_> cwilso: add a "send space available" call
<ChrisL_> ... not a "wait until this much space is available"
<ChrisL_> cwilso: stream implementations are uncommon as yet
<ChrisL_> mdjp: what about doing the minimal for v1 and
re-evaluate for v2?
<ChrisL_> cwilso: that is one way to do it
<ChrisL_> ... remembering how windows api works here, is it
dynamically reallocating
<ChrisL_> ... duplication of effort
<ChrisL_> ChrisL_: prefer the minimal backpressure for v1,
close issue, re-open only if new data shows that is not
sufficient
<ChrisL_> cwilso: ok, need to talk to the TAGG and see what
they think
<ChrisL_> ... WebUSB and WebBT do not expose backpressure, they
have an async promise based send model
<ChrisL_> cwilso: rest of the issues are editorial
<mdjp> [21]https://github.com/WebAudio/web-audio-api/issues/251
[21] https://github.com/WebAudio/web-audio-api/issues/251
<ChrisL_> mdjp: ok, great. now nto the webaudio issue
<ChrisL_> (cwilso reads)
<ChrisL_> cwilso: think that joe did a subclassing test which
failed because of #250. Have to go through create methods o
audioContext. once fixed, subclassing works as well
<ChrisL_> ... can we compose an audio node and hook up a bunch
of other things, an then call connect onto an arbitrary point
in the graph.
<ChrisL_> cwilso: can't do constructible audio params
<ChrisL_> ... think this does work today, crate node and
override .connect
<ChrisL_> (cwilso tests)
<ChrisL_> cwilso: yes, you can override it
<ChrisL_> ... cant expose as audio params
<ChrisL_> cwilso: being able to new and audio param and have it
run
<ChrisL_> cwilso: closed it because we put it on audio worker
<ChrisL_> mdjp: so it can stay closed
<ChrisL_> #134
<ChrisL_> #367 is postponed to v.2
<ChrisL_> hongchan: so in terms of subclassing we don't need to
change anything
<ChrisL_> cwilso: right, but composition does not, yet
<ChrisL_> cwilso: audio param is still magic. not newable.
marshalled across to worker behind the scenes
<ChrisL_> BillHofmann: would #250 solve that?
<ChrisL_> padenot: it could. yesterday we talked of making an
audio param from iside a worklet
<ChrisL_> Chair: Matt
<ChrisL_> Meeting: Audio WG f2f2, Atlanta, Day 2
<ChrisL_> cwilso: or we could say #251 and say 367 back
<ChrisL_> cwilso: don't care exactly because we need to keep
going on constructible params, whether it hits v.1 or not
<ChrisL_> padenot: we should get constructible audionodes
first, then constructible audio params, look at this issue.
<ChrisL_> rtoyg: agree
<ChrisL_> jdsmith: general question - there like 80 issues
ready for editing, this one is useful but if we wanted to get
these down to CR in summer, is this where we would be working
and push to v.next?
<cwilso> (Sorry, connection froze)
<ChrisL_> jdsmith: maybe we need a v1-nonblocking flag
<cwilso> I think this is pretty straightforward to spec; I
think the question of how hard it is to implement might factor
in there, but it is one of those fundamental,
Extensible-Web-Layering type issues.
<cwilso> hah
<ChrisL_> BillHofmann: Jerry asked if we should stop adding new
stuff and deal with the easier v.1 issues, pushing others to
v.2 or v.1.nonblocking ie desired but at risk
<ChrisL_> cwilso: it is an extensible web layering issue
<ChrisL_> .. avoiding magic
<ChrisL_> ... without this we can't extend
<ChrisL_> ChrisL_: agree this is architectural
<ChrisL_> padenot: if we realy want composite nodes, we need
audio param remapping
<ChrisL_> cwilso: useful when making a unity node, or a looping
single sample; other one is four params exposed on a composite
node
<ChrisL_> padenot: what about a composite audio param that
connects to two audio params
<ChrisL_> cwilso: right, you need to be able to do that
<ChrisL_> cwilso: title needs to be "constructible and
connectable"
<ChrisL_> mdjp: so, are we in v.1 or v.next?
<ChrisL_> padenot: we keep adding stuff to v.1. Reluctant to
but this is important
<ChrisL_> padenot: review backlog next week and triage
<ChrisL_> rtoyg: fair amount of design work in this one, too
<ChrisL_> rtoyg: not clear how it is supposed to work
<ChrisL_> mdjp: put a time tlimit on investigating this. Likely
a v.1 requirement.
<ChrisL_> mdjp: lets discuss in two weeks (no call next week)
<ChrisL_> mdjp: agenda is pretty much cleared now
<ChrisL_> Chair: Matt
<ChrisL_> Meeting: udio WG f2f2, Atlanta, Day 2
<ChrisL_> Meeting: Audio WG f2f2, Atlanta, Day 2
wrap-up and next steps
<ChrisL_> rtoyg: we have 13 open pull requests
<ChrisL_> padenot: will look at those next week
<ChrisL_> [22]https://github.com/WebAudio/web-audio-api/pulls
[22] https://github.com/WebAudio/web-audio-api/pulls
<ghaudiobot> [web-audio-api] rtoy closed pull request #770: Fix
#769 by correcting the formulas
(gh-pages...769-fix-lowpass-highpass-formulas)
[23]https://github.com/WebAudio/web-audio-api/pull/770
[23] https://github.com/WebAudio/web-audio-api/pull/770
<ChrisL_> meeting adjourned
Summary of Action Items
[NEW] ACTION: ChrisL to draft new charter [recorded in
[24]http://www.w3.org/2016/04/08-audio-minutes.html#action01]
[NEW] ACTION: ChrisL: to create implementation report [recorded
in
[25]http://www.w3.org/2016/04/08-audio-minutes.html#action02]
[NEW] ACTION: joe to liaise with the media capture and streams
to discuss options for adding audio and video device constraint
and enumeration, in context of existing group or the incubator
group [recorded in
[26]http://www.w3.org/2016/04/08-audio-minutes.html#action04]
[NEW] ACTION: mdjp to coordinate new use cases [recorded in
[27]http://www.w3.org/2016/04/08-audio-minutes.html#action03]
[24] http://www.w3.org/2016/04/08-audio-minutes.html#action01
[25] http://www.w3.org/2016/04/08-audio-minutes.html#action02
[26] http://www.w3.org/2016/04/08-audio-minutes.html#action04
[27] http://www.w3.org/2016/04/08-audio-minutes.html#action03
Summary of Resolutions
[End of minutes]
__________________________________________________________
--
Chris Lilley
@svgeesus
Technical Director, W3C Interaction Domain
Received on Friday, 8 April 2016 19:24:38 UTC