[![W3C][1]][2]

# Media Capture Task Force F2F at TPAC 2012

## 30 Oct 2012

[Agenda][3]

See also: [IRC log][4]

## Attendees

Present

    Adam_Bergkvist, Anant_Narayanan, Cullen_Jennings_(fluffy),
Dan_Burnett_(burn), Dominique_Hazael-Massieux, Eric_K_Rescorla, Giri_Mandyam,
Harald_Alvestrand, JeromeMarcon, Josh_Soref, Justin_Uberti, Martin_Thomson,
Matthew_Kaufman, Milan_Patel, Richard_Tibbett, Stefan_Hakansson,
Timothy_Terriberry_(derf), Travis_Leithead, natasha, Jim_Barnett

Regrets


Chair

    stefanh, hta

Scribe

    Milan, Josh_Soref

## Contents

  * [Topics][5]

    1. [Agenda bashing][6]

    2. [Constraints and settings][7]

    3. [Recording][8]

  * [Summary of Action Items][9]

* * *

<trackbot> Date: 30 October 2012

<Milan> scribe: Milan

### Agenda bashing

Candidate for discussion: Handling of multiple streams

Another candidate: how setting properties on one track affects derived streams

another candidate: when to turn on the "green light"

more candidate: fake (includes placeholder) streams

stefanh: what should we remove from the agenda?

anant: fingerprinting requires less than 15

... so may as well make that zero

hta: What does it mean to remove stream from PC? Answered with arrays are
stupid.

<dom> Dom: fingerprinting will be discussed tomorrow during plenary breakout,
whose time and place will be decided during agenda building between 9:30-10:30

Travis: getTrackById() is how HTML works

... our groups needs to have multiple active video streams

stefanh, perhaps we start with constraints and settings

<dom> [getTrackById in HTML5][10]

Travis: How about synchronization of tracks in a media stream

Jim_Barnett: I have that in the recording discussion

### Constraints and settings

burn: Synchronization also needs to be included in the recording discussion

... First slide

... do we support runtime settings to v stream tracks

... and what are the implications for peer connections

stefanh: what kinds of changes?

burn: slide 2, settings proposal

<dom> [Proposal: Settings API Version 4][11]

burn: focusing on points of interest to WebRTC folks

... media streams are device centric

... getUserMedia() composed of tracks specific to the device type

... tracks have relevant properties built into them

... we'll get into the specifics of the properties later

... slide 3

... device discovery list by device type

... once a single device has been approved, all other in that class exposed

anant: what about permissions?

Travis: That's a good question that needs to be addressed

burn: when a device goes away, a handler is fired

... constraints can be applied to set of devices

... looking for a successful path

Travis: width.min and width.max instead of minwidth and maxwidth

anant: why do we need a trial?

... why not just apply

Travis: purpose of trial is to use same constraints as would be used live

... giving the developer the possibility for later selection

anant: Trial is most useful for permission check

... this is v4 spec

... does this simply boil down to convenience?

Travis: yes

anant: granting access to one device doesn't extend to others even in the same
class

... any extra functionality?

Travis: Aside from convenience, this allows a query for the number and
capabilities of devices

... which addresses the privacy concern

fluffy: this is an expected user scenario

<Josh_Soref> +1 to dom's point

ekr: Giving permission for one camera should not extend to another

anant: perhaps we should wait for permissions answer

<Josh_Soref> scribe: Josh_Soref

<ekr> To clarify, what I am saying is that I don't see the relationship
between being able to access a camera and being able to see the list of
cameras

dom: when the user is first prompted the UA can let the user select which
devices should be available

<dom> I think it is meant as a convenient shortcut, not as a logical
relationship, ekr

Josh_Soref: +1

dom: I wouldn't base my schedule on getting a clear outcome from tomorrow's
meetup

anant: either give list of devices up front, or don't

... simply giving the full list after a first prompt doesn't seem to achieve
anything

dom: it protects against drive by attacks

<martin> I'm not sure that I'm a fan of having multiple permission grants to
do something like multi-camera telepresence either

dom: if you provide some indication of a level of trust

... I agree it's a bit flimsy, but it isn't entirely out

derf: we just ad this discussion

... we changed our implementation to turn the green camera light as soon as
you call getUserMedia

... before we turn on the sink

... we don't want the user to be surprised when the web page is able to turn
the camera on

Travis: assuming they can trust your green light

derf: that's a different issue

... the problem is now, I get the stream, I have the list of devices

... I can switch to any of those cameras

... do I need to turn all of their green lights on?

dom: I think it's a shortcut

... there's a distinction between green lights and chrome indicators

... the distinction between green light and chrome indicator matters here as
well

fluffy: re: dom

... there are two different permissions

... permission to look at the list

... permission to look at the camera

dom: asking twice

... in most cases the user doesn't care

fluffy: yes, but you're asking us to ask twice

anant: he's saying the second permission implies the first

fluffy: that's a confusing experience to the user

... click yes, you can see the list

... but before you can't

dom: I don't disagree

... I don't want to explain to my mother

... I'm not quite convinced it's the right approach

... it isn't a totally random approach

fluffy: there's two different permissions

... how many times do we ask user to click yes

gmandyam: going back to before getUserMedia

... you have a video facing/front facing

... I have an app

... that needs use of a front facing camera

... user gave me env facing camera

... I go back to user "this app isn't going to work"

... user is prompted again

... gives front facing

... I've determined he's got two

anant: how did you determine

gmandyam: with a change to the stream that indicates what it is

... my proposal had the ability to ask for front facing in a constraint

... with this, you got the user to disclose both cameras

... -- this way you got the user to give the same info

... I think it's a shorter way to get to the same result

Travis: before says something I don't like

... once you got the device list, it's fair game to do anything you want

... there will be a v5 that says "you can get the device list"

... but to turn it on, you use getUserMedia

... v5 is same permission requests

gmandyam: existing goes through same count of requests

burn: obtaining access to a media stream means you get info about the stream

... front/rear width/height

<dom> [granting access to the list of devices could be optional, with a
parameter in getUserMedia, with a checkbox in the authorization request]

Josh_Soref: so

... my preference is than when the first call to gUM is made, the user sees a
list of devices

... before I click OK on any camera I can see a preview of the camera

... if I tap on one, and it doesn't show me the correct preview, I'll click
another camera

... we should try to design this that the app doesn't have to ask twice

... the could be cases where you need additional cameras

... the user selects one plus a set of other granted cameras

... the app gets the granted device with an additional list of other granted
devices

<fluffy> josh: the gent user media interface should show the user a preview
windows that allows choosing the correct camera

Travis: I'd implement that

<martin> there was more to it than that, the idea of a multi-select for "the"
camera and several other cameras was proposed

ekr: my experience w/ apps is there's a need for the app to have camera
switching

... hangouts

... webex

... there's a need to enumerate

... there's a question of whether there's a need for permission

... I've heard "none are needed"

... "permissions are needed"

... "implicit permissions"

... I'd vote for no permissions to enumerate

... permission to activate

anant: every switch requires permissions?

ekr: every new acquisition requires permission

anant: Josh_Soref 's thing only works for front/back

... that doesn't work for other cases

... front/back is so useless

... cameras can rotate 360 degrees

... I think constraints are important

... you can't rely on users to pick the right device

... if fingerprinting isn't an issue, I +1 ekr

... for UCs, do you need a device name / constraints

... or just a count of video/audio

... I'd argue for minimal info

<richt> front/back is useful _relative to the device frame_. That's
irrespecitve of whether you can rotate any given camera 360 degrees or not.

anant: they'll know about constraints when they call getUserMedia

... they can assign labels/ids - I don't care

... we-mozilla have a prototype

... where you pass device:n to get info for that device

... we use that for our own chrome

adambe: fluffy, you had a good point

... an app asks, wants permission for a camera

... but you really want a list

... that's silly

... perhaps it would make sense to let the app ask to enumerate

... and then it could actually, from the list, find a camera

... and prompt the user, show self view

... I don't think previous discussion about this in WhatWG

... getting something + clicking ok

... you had to do an active selection

... in a file dialog

... you're browsing

... just clicking ok isn't really safe

... we don't want apps to request a dummy to enumerate

... I think it'll be pretty common

dom: enumeration of hardware isn't the only fingerprinting issue

... there's also hardware class

... what Josh_Soref said earlier

... to link the call to getUserMedia to granting access to several cameras at
once

... UI work on doing that is non trivial

... I think it's a good idea

... what adambe said,

... I want access to a camera, but also a list to other cameras on this
hardware

... and have a ui for a checkbox "let this app have info about list on this
device"

[ Slide Travis's settings proposal v4 ]

burn: specific value constraints

... instead of a setting is

... max/min together

... to give one value

... you allow max+min

... but focus is on this value

... width=value

... enumerated list = value

... different from constraints which have ranges

... or it's constraints where max=min

... settings override constraints

... it doesn't matter what was set before, you set something new

... and that's what can be adopted

... let's keep this short

hta: what I liked about v4

... I think I can see how to map them onto constraints

... so we have an underlying self consistent constraint based layer

... I didn't see the logic picture

derf: or you get "pick closest value"

gmandyam: is the value change async

... with a callback

... settings aren't applied async, they're sync

... I set zoom/redeye

... I immediately make the call, and don't need a callback

... we need to understand if settings are async

<martin> can you set zoom atomically and immediately like that?

Travis: as an implementer, when you talk to random cameras

... if you make that sync

... it's a problem

... same problem as sync file system access

gmandyam: then you need to address that in v5

... we're ok with that

... my internal guys want async

[ demo of zoom taking time ]

anant: white balance is a discrete value

... you might benefit from constraints and not settings

... you'll know if it fails/succeeds

... if you specify mandatory on an existing stream

... you'll get a failure callback

Travis: or it turns off your camera

anant: I'd expect you to call the failure callback

... not turn off the camera

... we just define that

... my feeling is we offer getUserMedia like thing on track/stream

juberti: agree with anant

<ekr> +1 to anant, juberti

burn: currently in v4 proposal that's done on the device type specific track

... that's different from getUserMedia

... you could get any new device that satisfies constraints

anant: keep that

juberti: camera is open to resolution X

... you specify resolution Y

... does that crop/scale?

Travis: I don't know

juberti: I want both

richt: constraints/values in summary

... constraints are ranges

... this is about specific things

... but width/height is fishing blind

... it's a range

... settings is a responsive reaction to what's available

... it's much more focused on responding to user's hardware

fluffy: +1 on anant 's proposal

... a lot of constraints aren't ranges

... I always thought they encompassed settings types

... I think we always needed to be able to change things on a track

... some may feed to camera

... or do it internally if not

... I hope they don't scale up

<gmandyam> Picture specific settings should not have to be enumerated in the
RTCWeb Media Constraints doc

fluffy: can you have 2 tracks cloned off eachother at different resolutions
from the same camera?

Travis: +1

[ Impacts on Peer Connections ]

burn: settings proposal includes device-info

... not just track info

... info about what device is giving right now

Travis: there are a very small number of track level additions

... width/height not coming from device

... it's mostly device

burn: peer connections

... what happens

... if you have a video media stream track

... with specific width / height

... and you add it to a peer connection

... what can you expect to happen at other end

... when it arrives at other end

... is it still a video stream track

... could someone change it at the other end?

... currently. constraints in peer connection

... specific characteristics of media can be changed on the fly by browser as
necessary

... as long as they continue to satisfy the constraints

... for congestion control, whatever reasons the UA wants

... how does that work w/ or not w/ settings

... then device track swapping w/in stream

anant: I like removal of local-media-stream

... I think transmitter is always in control

... if you apply changes on one side

... [sender?]

... that should apply on the other side

... I think if you apply changes

... the UA can decide what to do

burn: application requests change where? and where is change applied?

anant: browser A getUserMedia camera

... sends camera to browser B

... browser A changes constraint for stream

... and is applied to stream seen in browser B

... on browser B

... if application there makes a constraint request change

... depends on what is asked for

... if it's a scale-down

... browser B can decide to do it

... in cases it can't

... call failure

... no device/peer connection

Travis: browser B receiving stream from peer connection

... it thinks peer connection is a source

... since it doesn't own source, it can do some operations

anant: yes, that's what I'm proposing

... scale down yes

... white balance maybe not?

stefanh: we discussed some of this yesterday

... what should happen if you modify media stream track

anant: when you specify a constraint request that violates the stream

... the stream should remain unchanged

... and call failure

adambe: what if browser is doing things to keep stream alive?

anant: unknown

adambe: about picture device track

... will those live on their own?

Travis: fuzzy in proposal

... to gauge reaction of group

... picture track could act like a video track

adambe: think it's helpful if it's coupled to a video device track

... something you can feature probe

... provides extra functionality

... inheritance

... properties containing others...

... [hand waving]

... seems it'd be better as an extra feature coupled to parent

<martin> +1 to what adam just suggested, PictureDevice is not a track

gmandyam: webrtc says remote streams are read only

... can't do things except on sender side

... not sure what to do if receiver isn't satisfied

... how does that work

... stream is readonly

<matthew> if I have synthetic tracks that come from files, wouldn't
picturedevice possibly represent a slide show of a folder of pictures (perhaps
not encodable as video)?

anant: failure callback

... browser can't make change

Travis: don't you need to put it in a <video>

anant: yes, but it's scaling down the video

gmandyam: so is it a concern?

anant: there are some kinds of constraints you can't apply

... if it's over the network

gmandyam: shouldn't that be part of SDP negotiation

anant: doesn't matter

... app is the same on both sides

hta: I have a proposal in IETF

... for signalling constraint change over SDP connection

... I have a proposal

... if there's language in the spec that says remote streams are readonly

... we need to change that language

... if we change incoming resolution

... what I want out of this is different

... the reasonable thing for video system to do

... is to rescale to requested resolution

... up being ugly

... and then negotiate which is a process that might take time

... I think it's reasonable to apply these constraints

... to a remote stream

... we should just allow that

... say to IETF

... (oh, hey, that's us)

... you need to figure out what needs to happen

fluffy: which constraints?

hta: resolution+framerate

fluffy: I think we have RFCs for that

gmandyam: you sent an email to RTCWeb

... receiver determines resolution isn't proper

... ideally it signals that

... SDP or whatever

... sender modifies

... which is going to the peer connection

hta: draft has 3 ways to do it

... one I recommend avoiding

... two describe what you're saying

... there's a little different for what the sender has to do

... your description applies to both

gmandyam: my goal is to avoid significant changes to getUserMedia

hta: doesn't affect that

burn: I like from local usage case

... I like specialized track types

... w/ extra info/settings

... I like setting settings via settings or constraints

... discussion we just had leaves me unconvinced we understand how they'll
work in the entire process

... applications can talk to eachother

... we have signaling and communications

... side B can say "i want better source video"

... UA doesn't need to do that

... application can

... and hta sent email to list

... there's a pipe metaphor

... stream peer stream

... there's a lot happening on the way

<hta> [http://alvesnotes.blogspot.fr/2012/03/byte-stream-fallacy.html][12]

burn: while in peer connection, you should be able to control constraints for
sending

... and on other side as well

... streams should be customized for local use

... streams should be customized for peer connection use

... it should have its own set of constraints

... which may succeed/fail

... don't make app do everything

... I'm not at all convinced it'll work right

<Zakim> Josh_Soref, you wanted to ask if browser B has a constraint and to ask
if browser B has a constraint and browser A changes constraint to violate B's

Josh_Soref: if browser B has a constraint, and browser A changes the
constraint

... what happens?

burn: you said that B had a min-width?

hta: start by scaling

... then in the middle we have SDP negotiation failure

... not sure what it says

... but we have an RFC for it

fluffy: really simple

... I demand left handed threads, you demanded right handed

... the stream will fail

... negotiation fails

burn: same as if user unplugs camera

hta: Ended event

stefanh: ended on track

matthew: I think using offer answer SDP

... will never work fully

... you offer, he answers, but things never match up

... eventually someone rejects

... seems insanely complicated

... let's do it!

... there's a rejected proposal which contains a solution

fluffy: there's comment 22

martin: who wrote it?

matthew: person to my right, ekr [you]

<martin> I wrote it, it's not a big deal

<ekr> I am ekr

<ekr> that is martin

<fluffy> when will comment 22 end

<fluffy> ?

<ekr> the guy in the hat is ekr

<martin> poor Josh

[ scribe note: they totally messed up that transcript ]

<ekr> the guy in the red stripy shirt is martin

matthew: user changes sizes

... see if you can even get SDP that converge

juberti: I think it's possible

burn: I wish we had 3 hours

... half hour for 6 major topics isn't enough

... is there anyone who can hazard what we agreed

<matthew> existing systems that do it use the media path to do it

fluffy: can't repeat anant 's proposal

... I think I agree w/ a fair amount of it

<matthew> and limit the changes to a few codec-related parameters

fluffy: how does this cross remotes

... I think there are a fair number who want remote to change res/framerate

... I would guess those two things would have widespread agreement for wanting

juberti: about remote changes

... they're suggestions

... mandatory isn't going to lead to good things

fluffy: +1

<matthew> consider the case of dealing with negotiating landscape vs portrait
when both users have devices that can rotate

burn: optional=hint

Travis: is there widespread agreement on approach of track types mapping to
devices?

hta: significant support for that

burn: Video, Audio, Picture?

Travis: that concept

<matthew> nobody sends an offer to send the video in the other orientation and
waits for an answer before rotating it/changing the aspect ratio

fluffy: I'm comfortable with that

<matthew> and that's for a very good reason

fluffy: as long as new types of media that aren't video/audio are possible

burn: +1 for that

<martin> the oo purist in me would like to remind that group that prefer
composition over inheritance is a good rule

Jim_Barnett: isn't that already today

burn: no, we need a future track for something else

fluffy: find one construct that works for both things

burn: fair number of people who'd like changes for res/framerate

... on sink, possibly resulting in auto changes on source

... hard to argue w/ people who want that

... I'm concerned it'd really work

... not objecting

fluffy: don't want to say "such as"

... say res + framerate -- only

... maybe not possible

... play around, might be able to make it possible

burn: constrain set sufficiently

anant: if you only make a request

... why doesn't the app send a message to itself

... instead of SDP

hta: can RTCWeb have agenda time for it next week

fluffy: no

... we'll still be discussing it next summer

... to anant 's question

... 3 layers

... out of band (xhr)

... sdp

... with several codec specific, and a draft from hta

... and people who say it happens in rtc-rtp

... perhaps if we knew which layer was doing it

anant: I think it's easiest in the highest

fluffy: slowest too

anant: true

fluffy: progress is being made

... hta is writing drafts

burn: in order to get the flexibility for apps

... keep adding layers at lower level

... which has to reflect at other layers

... then eventually you do it at the higher level

matthew: other reason is if you really like SDP

Travis: when you apply a constraint via getUserMedia to a device

<matthew> (which we must here, otherwise we'd have made another decision some
months ago)

Travis: does group have agreement that that defines the valid range of
settings at any time after

... or not

... do you have to turn off app/turn back on to reset?

hta: if we model change of constraints

... then you can logically change limits

... but if you move outside device support, then setting fails

... on list, we discussed

... if browser UA decides to change params

... without consulting w/ app

... then it should only change w/in current constraints

stefanh: we don't know what to do if it must move outside

hta: mac cameras drop framerate

Travis: anant and I sync up on changing settings w/ constraints

dom: originally getUserMedia was near LC

burn: not this week

dom: doesn't need well defined

... just media stream hierarchy

fluffy: I'd guess a year

dom: you create one document w/ what's stable

burn: 5 months, optimistic

fluffy: I'll go for that, very optimistic

martin: you said "automatic"

... you set constraints on receiving end

... that concerns me

... you said automatically send to sender

burn: that's what I'm understanding

... may magically

... things propogate magically

Josh_Soref: recieving side wants big frames NOW, don't wait for remote stream.

Jim_Barnett: you have a fair amount of control by doing things in <video>

... and say everything else is done to apply across the wire

matthew: if we don't have your draft

... then you could do it out of band (not choice)

[ Constraints / runtime / coffee time ]

burn: I'll run through, no questions

[ hta's video proposal constraints - IETF ]

[ Travis's proposal - device settings ]

Travis: dimension = width+height at the same time

[ gmandyam 's proposal picture constraint ]

burn: thanks Josh_Soref for scribing

hta: we want to open the speaker queue

... people have had a few minutes to think about what we talked about before
the break

<martin> +1

hta: it was suggested that we have a five day interim and not let people out
before we have decided

... (joke)

... are there question or comments on this topic?

(topic is: what constraints/setting- do we support)

fluffy: there are a lot of things to set on a camera

hta: there could be proprietary ones

fluffy: what do user understand when it comes to settings?

Travis: get v1 out the door

... with a minimal set

<fluffy> hi

<adambe> test

burn: Was strongly encouraged to name it for ownership by one group.

<adambe> (scribe lost connection)

<martin> gmandyam: not keen on going to two bodies to get values added

fluffy: it's not a IETF specific registry (constraints)

burn: years ago.. IEFT and W3C didn't work together on registries

fluffy: any organization can modify the registry

dom: if we want interop we need to test the constraints

gmandyam: v2 of document may be fiction

fluffy: burn, what's the process for the registry

hta: I think it was "expert review"

burn: the entries are registered in the registry, but they are defined in the
w3c spec

gmandyam: living document is a red herring

... I think we need to pick a set of constraints for v1

hta: I don't think you can modify constraints

gmandyam: I meant to modify the list

<fluffy> The current registry is Specification Required with Expert Review"
registration policy as defined in RFC5226 [RFC5226].

Jim_Barnett: if a constraints isn't documented in w3c, where is it registered?

burn: there's an IANA section in both documents

<fluffy> The registry creation document is at [http://tools.ietf.org/html
/draft-burnett-rtcweb-constraints-registry-02][13]

Jim_Barnett: so between v1 and v2 you couldn't add constraints?

burn: a w3c document specifies which constraints from the registry that needs
to be implemented to be compliant with the spec

<fluffy> The other relevant spec is [http://tools.ietf.org/html/rfc5226][14]

burn: the registry should prevent organizations from registering general names
that we might want to have for "official" use later

<martin> [http://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-
capture/proposals/SettingsAPI_proposal_v4.html][11]

<martin> the above contains a proposal for the initial set of constraints;
provide comments on this

anant: where was a question about which constraints we should support for gUM

... we should have one list

<martin> gmandyam: you would be able to attempt to set constraints, but these
settings would fail

(scribe got thrown out)

discussion about mic input level

juberti: we could skip mic input level in v1

... AGC solves a lot of cases

fluffy: I agree with juberti

burn agrees with juberti as well

[ v5 that anant and travis agree on will be sent out ]

<dom> ScribeNick: Josh_Soref

### Recording

hta: does anyone remember the structure of errors?

... a number plus string

anant: it should be a string

hta: I got a question from my impl team

... they want more failures beyond permission denied

... like no camera

juberti: no camera, failed, permission denied

Travis: constraints not happy

juberti: hardware not happy

<burn> FYI, media constraints registry policy is "Specification Required with
Expert Review"

anant: current getUserMedia has only one error

... it should have more

... for constraint API, it should have a single error kind

hta: no camera, camera malfunction

fluffy: it's about who gets the support call

hta: objection to adding these three codes?

Jim_Barnett: if t1 and t2 are in stream 1, they're sync in that context

... but if t1 is in s 2 and t2 is in s3, they aren't synced?

derf: they're probably sync'd in our current impl

... t2 in a second video window may be delayed

Jim_Barnett: and sync'd w/ different audio if they have different starting
points

... derf no container format allows adding and removing of tracks while
providing stream encoding

... there was a request to add/drop streams as you go

fluffy: this makes sense

... I've wondered why

... why do we deal w/ container format

... inside the browser

... why don't we allow the browser to construct the container format

<martin> I propose that we rename MediaStreamTrack to MediaStream and
MediaStream to MediaStreamSync

fluffy: I'm worried about the topic of choosing a container

Jim_Barnett: if you want a recording, it needs to be in a container

... so it can be played back

fluffy: the browser needs to provide the raw data

... and someone needs to be able to save it to disk

anant: what do you mean raw data?

... pcm?

matthew: flv?

fluffy: you know I have an answer

... I'd probably go w/ a compressed version of the data

derf: w/ compressed data, you need timestamps

matthew: flv

derf: backreferences

martin: you end up w/ a container format

Jim_Barnett: it's a requirement that the base case is simple

... so it should be an appropriate container

fluffy: I know how to solve it

... but all of the containers are workable

derf: that's mostly naive

fluffy: not too naive

... if we're going down a path where the browser returns a container format

... it isn't useful unless we specify which container format the browser uses

derf: you have canvas.getDataURL with a mimetype

... the browser passes back data in that mime type

... or I don't support that

fluffy: is there a mandatory to implement set?

hta: reference normatively the media source spec

... and reference whatever they mandatorily implement

fluffy: they don't have one

... they don't intend to get one

Travis: all the better

derf: there's an open bug for that

fluffy: do you believe they intend to fix it?

... the point of recording is to be able to do something w/ the data

Jim_Barnett: as long as the browser can play it back?

Josh_Soref: if you can take the data and have the browser reproduce it as a
stream source

... which can be sent to <video> (with extract via draw to canvas / and
imaginary audio API),

... or sending to a peer stream

fluffy: I agree with you

... there should be an API to get raw access to the data

... I don't think it should be this API

hta: media source API has a goal to allow byte stream access

[ hta is reading the spec requirements ]

fluffy: the draft does what I suggested

... which is the opposite of what they said in the room

Travis: are you recording to MPEG4 or akin to that

fluffy: something like that

... I suspect half vendors are thinking one open source format

... unlikely to be supported by the other vendors

... making it largely useless to most people

Travis: having only one browser in the world would make everyone's life easier

fluffy: have raw audio samples

... can wrap in container A or B

... wrapping can be done by JS

... why not just give the raw audio samples?

Jim_Barnett: how do you sync?

anant: I agree on PCM

... but what do you do w/ raw video?

martin: Frame, Frame, Frame

... how do you get pixels off canvas?

anant: can you implement this in js?

Josh_Soref: sure

fluffy: there's a WebEx does H264 in JS

[ ekr has a 1x1 codec ]

[ laughter ]

fluffy: you wouldn't want to re-encode video if you had them that way

derf: I'd rather give you the

fluffy: if you gave me the data and RTP time packets

... that's all I need

... I wouldn't really want that API, but it's provably possible

derf: I agree it's possible

anant: if you get the data over peer connection

... we've agreed on the MTI codec

... if we can agree on the codec

fluffy: sure

anant: peer connection between IE and Firefox w/ VP8 or H264

... then the container isn't a big deal

dom: I'd like a recording API today

anant: if we don't have MPEG today

... I'd like an API that lets the app ask for a format, and try

... that's how the web works today

<martin> no one wants my MTI codec : [http://tools.ietf.org/html/rfc2435][15]

derf: I think that's how the API should work regardless of codec or container

Travis: I agree w/ derf+anant

fluffy: ok

[ Recording ]

Jim_Barnett: sync'd recording at media stream

... not at track level

... and people want to add media stream as you go

adambe: you could squeeze in media stream constructor

Jim_Barnett: got a media stream to a <video>

... and you want to record what the user is seeing

<matthew> I suggested MJPEG yesterday, didn't I?

adambe: to record what you're seeing in <video>

... you could record that

Jim_Barnett: you're going to want to use a media stream you're already using

martin: if you have stream, then you can do track by wrapping it in a stream
of just that

Jim_Barnett: yes, but not an API for tracks directly

... encoding function for stream at interval

... another for whole blob

... stream encoding must ignore added tracks

... per derf, no container format supports this

... remove a track, provide silence/black

... probably have to fake fourth track

Josh_Soref: couldn't you cheat and always have 15 dummy tracks?

Jim_Barnett: there's no way to do that today

... that'd make it work

matthew: multiple things raised in the last 45s

... you can't add tracks

... and you can't do anything to them?

derf: changing res/frame rate is generally not a problem

... some formats don't support it

matthew: what about dummies

Jim_Barnett: suggestion was dummy tracks

matthew: we need dummy tracks anyway

Jim_Barnett: I didn't think we had them

matthew: it's supposed to be on the todo list

Jim_Barnett: we could say you can do anything the ua can do

... but if we know, we should state up front

... it's always possible to fail

... changing res

... if that fails in some format

derf: I can make it work, by rescaling to dest res

matthew: or just make everything one pixel

fluffy: it looks like advanced authoring format does allow it

... more accurate statement is

... "the one format we want to use will support it doesn't support it"

matthew: mp4

... it's true if you write it up front

... but not at the back end

derf: I don't think things will play it back correctly

matthew: that's different

derf: they won't play

fluffy: what if you write it at front and back patch

derf: if I know up front, I can

[ cross between fluffy and derf about supported ]

fluffy: windows players will support [something]

derf: I'd be shocked

Travis: does anyone object to the general ability to

matthew: mpeg2 transport stream definitely does

derf: I challenge you to find something that reliably supports it

... there's one format people hate more than ogg

... it's mpeg2 transport streams

Travis: does anyone object to specifying limits that you can't add/remove when
you're recording under stream conditions

matthew: I think you should get an error when the stream format doesn't
support X

... you could support a container format that doesn't support aspect ratio
change

... and the user changes

adambe: if I record, and the aspect ratio change, where is the error?

... I guess from the recorder

Jim_Barnett: the stream is valid

adambe: so the recorder needs to throw errors

derf: when you start recording, it records the tracks you had then

... and ignores others

matthew: why not allow it

... and let others do it in the future

fluffy: I like matthew 's phrasing

... an error from the recorder when it doesn't support x

martin: recording

... stopping when you get an error - sucks

matthew: you could have soft errors

... when you flip h/v

fluffy: question is stop or not

Jim_Barnett: streaming

... you return as much data as possible

... return error

... return new stuff as new format

derf: there's a way to do that

matthew: mpeg2 ts is like that

derf: if you stop all streams, and start recording a new set of streams

... that works fine in ogg

Jim_Barnett: streaming recording

... you / someone produces a change that produces unhandleable

... recorder produces an error

Josh_Soref: it should stop the data, send error, and unless you do
preventDefault() it should provide new recording automatically

derf: the larger error is the data is flowing async through your stream

... that you haven't seen

... if you start recording some time in the future

... anything in the future

anant: don't interrupt what was happening unless you absolutely have to

... the callback should have the option to let things continue

adambe: I think a specific API is better than that

anant: I'd say return true/false

Travis: that means we define these error events as being Cancelable

... with a specific behavior

martin: cancelable happens

... what happens if you cancel

hta: make sure next ver of proposal is clear about what happens

matthew: for orientation constraint

... can you prevent that from happening

Travis: my view differs from hta's

... and I'm slowly becoming a convert

... scenario

Josh_Soref: Scribe wants to record data coming in from peer connection

... and the peer connection input changed orientation

Travis: tracks have possible incompatible changes

... that may/may not affect recording

Jim_Barnett: blob recording

... we need to be clear about if you're asking for a Blob or asking for
Streaming

Travis: the blob container

... to overload the term

... blobs can be anywhere

<matthew> you flip your camera... the offer comes to me with the new aspect
ratio... I send answer accepting the new aspect, but in the meantime my user
has rotated their screen to see your video correctly... your user now sees
rotated video, so they flip their screen again, only they can't send an offer
because their offer is outstanding. meanwhile, my side sends an offer (it can,
because an answer was already sent)... so when your side goes to send an
of[CUT]

<matthew> now offer-offer glare.

[ Open Questions ]

martin: I like the property to prohibit multiple concurrent recordings

<matthew> if your users keep doing this for 11 hours, I can fly back to the US
from here to sooner than the O/A exchange converges

martin: as an implementation detail that would fail

Jim_Barnett: there's a UC for user is talking

... and you want to record to file and do speech transcription concurrently

hta: resource constraint

... makes sense to say starting a recording can fail

Jim_Barnett: oh yes

... you can get a failure at any time

hta: no need for a specific restriction

Jim_Barnett: not a specific constraint in the recorder

martin: there may be limitations in your implementation

Josh_Soref: these edge cases are nice to talk about here

... but if they're not that common it's likely that developers won't handle
them

... random failure is the only way to get developers to handle these kind of
rare errors

... there are no good solutions

martin: we could enable developers to trigger these them selves

Travis: it'd help if we talked in the document about how to trigger these
errors

... so that if you're reading this, you'll know that you need the error
handler

Jim_Barnett: I think from this, there's enough to write something

... is this a new document?

hta: from dom, it seems like a separate document

... for the time being

adambe: even streaming recording is using blobs

Josh_Soref: "at once recording"

... "chunked recording"

gmandyam: is this a v1 v2 feature?

Jim_Barnett: as a separate document, it isn't before v1

<matthew> "at most once"

gmandyam: simple media stream recording could be added to v1 and more advanced
recording could wait to v2

... media recorder in android and other mobile devices is pretty restricted

... we could do something similar in v1

... (i.e. simple version)

Travis: do you mean that this proposal would be v2 then?

<dom> scribenick: adambe

gmandyam: yes

hta: I think the described solution is pretty simple

fluffy: I'm interested in access to uncompressed data

(that was a question to derf)

derf: we had an idea about a MediaStream processing API

... but then we went for the Web Audio API

... which doesn't handle video

stefanh: I think we're done with recording

... next item is identity tagged media streams

gmandyam: I wanted that discussion to take place in the MC TF

ekr: I'll make sure that the comments are forwarded to both lists

stefanh: let's move on

... next topic - generating fake tracks

fluffy: I want to create an object similar to what you get from gUM

... it might be a 1x1 pixel video

(needs license from ekr)

matthew: I want to attach video to a PeerConnection so it's included in the
negotiation

... when I have real access to devices - I swap out the dummy

... for audio, if we don't fix this we will have a lost "hello"

... people will click OK and say hello

... there's no transport ready

... I don't care how we solve this

... silent audio, black video

derf: you could use a processed media stream

... you could feed the output from a video tag to it

matthew: it's too complicated for my case

gmandyam: you could generate comform noise

derf: it's similar

matthew: this could maybe work

... soft phones will suck if you can't provide fake streams (which can be
swapped for the real thing)

stefanh: I think we need this

... someone should draft an API for it

fluffy: how do I take an web audio object and create a media stream from it

... ?

derf: I don't know if it's in the spec (scribe missed some stuff here)

fluffy: if the default constructor created a empty silent stream

... we should define such an object

martin: a problem is that we can't connect that to the peerIdentity solution

gmandyam: so the problem is that the app is not trusted?

(several people); yes

Travis: is the purpose for fake streams a promise for a stream in the future

matthew: it has to look like stream from a camera and a mic

... I don't want to renegotiate

hta: so you wan't to send the new stream on the same m-line with same SSRC

matthew: the problem is that the session is receive only for the party being
called

... how do I respond to a rend-recv if I don't have anything to add?

fluffy, ekr: could this be solved with pr-answer?

matthew: no

[ martin draws on board ]

[ cullen is drawing ]

[ discussion: if the A side receives data after it has offered media to the B
side. Can it do something useful with it -- before getting the answer? ]

[ white ink on white board :) ]

<fluffy> @dom, I think we are getting closer back to w3c land

hta: trying to summarize

… we are trying to get the information to construct an answer for the SSRC for
the video that will be added later when the user consents

… suggestion is to have getUserMedia immediately return something that can be
added to PeerConnection before there is consent

stefanh: we have no way for caller to reject some of the streams the callee
wants to send

juberti: clue folks are looking at 4 way handshake for this

… clue people are still working on this

anant: the two objections to make gUM sync: 1) breaks existing applications 2)
Imposes bad user experience

<martin> I'll summarize the proposal: getUserMedia immediately returns a
MediaStream with the requested tracks. These tracks are fake until consent is
acquired or consent is denied (which causes the track to end).

ekr: people will set the "promise object" to a video src directly without
using the success/error events

richt: You're proposing something Promisee-like that isn't Promise compliant.
Promises require .then callbacks with a success and error callback. So if we
do this we do that. And we already have that already (minus the sync return
object).

... i.e. we already have success and error callbacks, just not wrapped in
.then.

ekr: the existing callback forces the developer to wait until a valid stream
is returned before using it

... the new proposal doesn't enforce this

<richt> so it's either mediaStream = getUserMedia(opts, promiseSuccess,
promiseError) or mediaStream = getUserMedia(opts).then(promiseSuccess,
promiseError) ?

anant: the result from gUM doesn't need to be a "real" MediaStream

juberti: I don't like that this break existing code

... a new suggestion: let gUM return an object and fire the callback as well

... then we wouldn't break existing code

<richt> +1 to jubertI's proposal as I explained above.

burn: we're not in last call

... we will break code up until we get to last call

hta: I propose a gUM2()

martin: two ways to do something is really bad

hta: I think we understand the problem

stefanh: we have two problems

martin: num_problems > 1

... is true

end of meeting!

puh!

## Summary of Action Items

[End of minutes]

* * *

Minutes formatted by David Booth's [scribe.perl][16] version 1.135 ([CVS
log][17])

$Date: 2009-03-02 03:52:20 $

   [1]: http://www.w3.org/Icons/w3c_home

   [2]: http://www.w3.org/

   [3]: http://www.w3.org/wiki/Oct_30_2012

   [4]: http://www.w3.org/2012/10/30-mediacap-irc

   [5]: #agenda

   [6]: #item01

   [7]: #item02

   [8]: #item03

   [9]: #ActionSummary

   [10]: http://dev.w3.org/html5/spec/media-elements.html#dom-audiotracklist-
gettrackbyid

   [11]: http://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-
capture/proposals/SettingsAPI_proposal_v4.html

   [12]: http://alvesnotes.blogspot.fr/2012/03/byte-stream-fallacy.html

   [13]: http://tools.ietf.org/html/draft-burnett-rtcweb-constraints-
registry-02

   [14]: http://tools.ietf.org/html/rfc5226

   [15]: http://tools.ietf.org/html/rfc2435

   [16]: http://dev.w3.org/cvsweb/~checkout~/2002/scribe/scribedoc.htm

   [17]: http://dev.w3.org/cvsweb/2002/scribe/