See also: IRC log
<trackbot> Date: 30 October 2012
<Milan> scribe: Milan
Candidate for discussion: Handling of multiple streams
Another candidate: how setting properties on one track affects derived streams
another candidate: when to turn on the "green light"
more candidate: fake (includes placeholder) streams
stefanh: what should we remove from the agenda?
anant: fingerprinting requires less than 15
... so may as well make that zero
hta: What does it mean to remove stream from PC? Answered with arrays are stupid.
<dom> Dom: fingerprinting will be discussed tomorrow during plenary breakout, whose time and place will be decided during agenda building between 9:30-10:30
Travis: getTrackById() is how HTML works
... our groups needs to have multiple active video streams
stefanh, perhaps we start with constraints and settings
<dom> getTrackById in HTML5
Travis: How about synchronization of tracks in a media stream
Jim_Barnett: I have that in the recording discussion
burn: Synchronization also needs to be included in the recording discussion
... First slide
... do we support runtime settings to v stream tracks
... and what are the implications for peer connections
stefanh: what kinds of changes?
burn: slide 2, settings proposal
<dom> Proposal: Settings API Version 4
burn: focusing on points of interest to WebRTC folks
... media streams are device centric
... getUserMedia() composed of tracks specific to the device type
... tracks have relevant properties built into them
... we'll get into the specifics of the properties later
... slide 3
... device discovery list by device type
... once a single device has been approved, all other in that class exposed
anant: what about permissions?
Travis: That's a good question that needs to be addressed
burn: when a device goes away, a handler is fired
... constraints can be applied to set of devices
... looking for a successful path
Travis: width.min and width.max instead of minwidth and maxwidth
anant: why do we need a trial?
... why not just apply
Travis: purpose of trial is to use same constraints as would be used live
... giving the developer the possibility for later selection
anant: Trial is most useful for permission check
... this is v4 spec
... does this simply boil down to convenience?
Travis: yes
anant: granting access to one device doesn't extend to others even in the same class
... any extra functionality?
Travis: Aside from convenience, this allows a query for the number and capabilities of devices
... which addresses the privacy concern
fluffy: this is an expected user scenario
<Josh_Soref> +1 to dom's point
ekr: Giving permission for one camera should not extend to another
anant: perhaps we should wait for permissions answer
<Josh_Soref> scribe: Josh_Soref
<ekr> To clarify, what I am saying is that I don't see the relationship between being able to access a camera and being able to see the list of cameras
dom: when the user is first prompted the UA can let the user select which devices should be available
<dom> I think it is meant as a convenient shortcut, not as a logical relationship, ekr
Josh_Soref: +1
dom: I wouldn't base my schedule on getting a clear outcome from tomorrow's meetup
anant: either give list of devices up front, or don't
... simply giving the full list after a first prompt doesn't seem to achieve anything
dom: it protects against drive by attacks
<martin> I'm not sure that I'm a fan of having multiple permission grants to do something like multi-camera telepresence either
dom: if you provide some indication of a level of trust
... I agree it's a bit flimsy, but it isn't entirely out
derf: we just ad this discussion
... we changed our implementation to turn the green camera light as soon as you call getUserMedia
... before we turn on the sink
... we don't want the user to be surprised when the web page is able to turn the camera on
Travis: assuming they can trust your green light
derf: that's a different issue
... the problem is now, I get the stream, I have the list of devices
... I can switch to any of those cameras
... do I need to turn all of their green lights on?
dom: I think it's a shortcut
... there's a distinction between green lights and chrome indicators
... the distinction between green light and chrome indicator matters here as well
fluffy: re: dom
... there are two different permissions
... permission to look at the list
... permission to look at the camera
dom: asking twice
... in most cases the user doesn't care
fluffy: yes, but you're asking us to ask twice
anant: he's saying the second permission implies the first
fluffy: that's a confusing experience to the user
... click yes, you can see the list
... but before you can't
dom: I don't disagree
... I don't want to explain to my mother
... I'm not quite convinced it's the right approach
... it isn't a totally random approach
fluffy: there's two different permissions
... how many times do we ask user to click yes
gmandyam: going back to before getUserMedia
... you have a video facing/front facing
... I have an app
... that needs use of a front facing camera
... user gave me env facing camera
... I go back to user "this app isn't going to work"
... user is prompted again
... gives front facing
... I've determined he's got two
anant: how did you determine
gmandyam: with a change to the stream that indicates what it is
... my proposal had the ability to ask for front facing in a constraint
... with this, you got the user to disclose both cameras
... -- this way you got the user to give the same info
... I think it's a shorter way to get to the same result
Travis: before says something I don't like
... once you got the device list, it's fair game to do anything you want
... there will be a v5 that says "you can get the device list"
... but to turn it on, you use getUserMedia
... v5 is same permission requests
gmandyam: existing goes through same count of requests
burn: obtaining access to a media stream means you get info about the stream
... front/rear width/height
<dom> [granting access to the list of devices could be optional, with a parameter in getUserMedia, with a checkbox in the authorization request]
Josh_Soref: so
... my preference is than when the first call to gUM is made, the user sees a list of devices
... before I click OK on any camera I can see a preview of the camera
... if I tap on one, and it doesn't show me the correct preview, I'll click another camera
... we should try to design this that the app doesn't have to ask twice
... the could be cases where you need additional cameras
... the user selects one plus a set of other granted cameras
... the app gets the granted device with an additional list of other granted devices
<fluffy> josh: the gent user media interface should show the user a preview windows that allows choosing the correct camera
Travis: I'd implement that
<martin> there was more to it than that, the idea of a multi-select for "the" camera and several other cameras was proposed
ekr: my experience w/ apps is there's a need for the app to have camera switching
... hangouts
... webex
... there's a need to enumerate
... there's a question of whether there's a need for permission
... I've heard "none are needed"
... "permissions are needed"
... "implicit permissions"
... I'd vote for no permissions to enumerate
... permission to activate
anant: every switch requires permissions?
ekr: every new acquisition requires permission
anant: Josh_Soref 's thing only works for front/back
... that doesn't work for other cases
... front/back is so useless
... cameras can rotate 360 degrees
... I think constraints are important
... you can't rely on users to pick the right device
... if fingerprinting isn't an issue, I +1 ekr
... for UCs, do you need a device name / constraints
... or just a count of video/audio
... I'd argue for minimal info
<richt> front/back is useful _relative to the device frame_. That's irrespecitve of whether you can rotate any given camera 360 degrees or not.
anant: they'll know about constraints when they call getUserMedia
... they can assign labels/ids - I don't care
... we-mozilla have a prototype
... where you pass device:n to get info for that device
... we use that for our own chrome
adambe: fluffy, you had a good point
... an app asks, wants permission for a camera
... but you really want a list
... that's silly
... perhaps it would make sense to let the app ask to enumerate
... and then it could actually, from the list, find a camera
... and prompt the user, show self view
... I don't think previous discussion about this in WhatWG
... getting something + clicking ok
... you had to do an active selection
... in a file dialog
... you're browsing
... just clicking ok isn't really safe
... we don't want apps to request a dummy to enumerate
... I think it'll be pretty common
dom: enumeration of hardware isn't the only fingerprinting issue
... there's also hardware class
... what Josh_Soref said earlier
... to link the call to getUserMedia to granting access to several cameras at once
... UI work on doing that is non trivial
... I think it's a good idea
... what adambe said,
... I want access to a camera, but also a list to other cameras on this hardware
... and have a ui for a checkbox "let this app have info about list on this device"
[ Slide Travis's settings proposal v4 ]
burn: specific value constraints
... instead of a setting is
... max/min together
... to give one value
... you allow max+min
... but focus is on this value
... width=value
... enumerated list = value
... different from constraints which have ranges
... or it's constraints where max=min
... settings override constraints
... it doesn't matter what was set before, you set something new
... and that's what can be adopted
... let's keep this short
hta: what I liked about v4
... I think I can see how to map them onto constraints
... so we have an underlying self consistent constraint based layer
... I didn't see the logic picture
derf: or you get "pick closest value"
gmandyam: is the value change async
... with a callback
... settings aren't applied async, they're sync
... I set zoom/redeye
... I immediately make the call, and don't need a callback
... we need to understand if settings are async
<martin> can you set zoom atomically and immediately like that?
Travis: as an implementer, when you talk to random cameras
... if you make that sync
... it's a problem
... same problem as sync file system access
gmandyam: then you need to address that in v5
... we're ok with that
... my internal guys want async
[ demo of zoom taking time ]
anant: white balance is a discrete value
... you might benefit from constraints and not settings
... you'll know if it fails/succeeds
... if you specify mandatory on an existing stream
... you'll get a failure callback
Travis: or it turns off your camera
anant: I'd expect you to call the failure callback
... not turn off the camera
... we just define that
... my feeling is we offer getUserMedia like thing on track/stream
juberti: agree with anant
<ekr> +1 to anant, juberti
burn: currently in v4 proposal that's done on the device type specific track
... that's different from getUserMedia
... you could get any new device that satisfies constraints
anant: keep that
juberti: camera is open to resolution X
... you specify resolution Y
... does that crop/scale?
Travis: I don't know
juberti: I want both
richt: constraints/values in summary
... constraints are ranges
... this is about specific things
... but width/height is fishing blind
... it's a range
... settings is a responsive reaction to what's available
... it's much more focused on responding to user's hardware
fluffy: +1 on anant 's proposal
... a lot of constraints aren't ranges
... I always thought they encompassed settings types
... I think we always needed to be able to change things on a track
... some may feed to camera
... or do it internally if not
... I hope they don't scale up
<gmandyam> Picture specific settings should not have to be enumerated in the RTCWeb Media Constraints doc
fluffy: can you have 2 tracks cloned off eachother at different resolutions from the same camera?
Travis: +1
[ Impacts on Peer Connections ]
burn: settings proposal includes device-info
... not just track info
... info about what device is giving right now
Travis: there are a very small number of track level additions
... width/height not coming from device
... it's mostly device
burn: peer connections
... what happens
... if you have a video media stream track
... with specific width / height
... and you add it to a peer connection
... what can you expect to happen at other end
... when it arrives at other end
... is it still a video stream track
... could someone change it at the other end?
... currently. constraints in peer connection
... specific characteristics of media can be changed on the fly by browser as necessary
... as long as they continue to satisfy the constraints
... for congestion control, whatever reasons the UA wants
... how does that work w/ or not w/ settings
... then device track swapping w/in stream
anant: I like removal of local-media-stream
... I think transmitter is always in control
... if you apply changes on one side
... [sender?]
... that should apply on the other side
... I think if you apply changes
... the UA can decide what to do
burn: application requests change where? and where is change applied?
anant: browser A getUserMedia camera
... sends camera to browser B
... browser A changes constraint for stream
... and is applied to stream seen in browser B
... on browser B
... if application there makes a constraint request change
... depends on what is asked for
... if it's a scale-down
... browser B can decide to do it
... in cases it can't
... call failure
... no device/peer connection
Travis: browser B receiving stream from peer connection
... it thinks peer connection is a source
... since it doesn't own source, it can do some operations
anant: yes, that's what I'm proposing
... scale down yes
... white balance maybe not?
stefanh: we discussed some of this yesterday
... what should happen if you modify media stream track
anant: when you specify a constraint request that violates the stream
... the stream should remain unchanged
... and call failure
adambe: what if browser is doing things to keep stream alive?
anant: unknown
adambe: about picture device track
... will those live on their own?
Travis: fuzzy in proposal
... to gauge reaction of group
... picture track could act like a video track
adambe: think it's helpful if it's coupled to a video device track
... something you can feature probe
... provides extra functionality
... inheritance
... properties containing others...
... [hand waving]
... seems it'd be better as an extra feature coupled to parent
<martin> +1 to what adam just suggested, PictureDevice is not a track
gmandyam: webrtc says remote streams are read only
... can't do things except on sender side
... not sure what to do if receiver isn't satisfied
... how does that work
... stream is readonly
<matthew> if I have synthetic tracks that come from files, wouldn't picturedevice possibly represent a slide show of a folder of pictures (perhaps not encodable as video)?
anant: failure callback
... browser can't make change
Travis: don't you need to put it in a <video>
anant: yes, but it's scaling down the video
gmandyam: so is it a concern?
anant: there are some kinds of constraints you can't apply
... if it's over the network
gmandyam: shouldn't that be part of SDP negotiation
anant: doesn't matter
... app is the same on both sides
hta: I have a proposal in IETF
... for signalling constraint change over SDP connection
... I have a proposal
... if there's language in the spec that says remote streams are readonly
... we need to change that language
... if we change incoming resolution
... what I want out of this is different
... the reasonable thing for video system to do
... is to rescale to requested resolution
... up being ugly
... and then negotiate which is a process that might take time
... I think it's reasonable to apply these constraints
... to a remote stream
... we should just allow that
... say to IETF
... (oh, hey, that's us)
... you need to figure out what needs to happen
fluffy: which constraints?
hta: resolution+framerate
fluffy: I think we have RFCs for that
gmandyam: you sent an email to RTCWeb
... receiver determines resolution isn't proper
... ideally it signals that
... SDP or whatever
... sender modifies
... which is going to the peer connection
hta: draft has 3 ways to do it
... one I recommend avoiding
... two describe what you're saying
... there's a little different for what the sender has to do
... your description applies to both
gmandyam: my goal is to avoid significant changes to getUserMedia
hta: doesn't affect that
burn: I like from local usage case
... I like specialized track types
... w/ extra info/settings
... I like setting settings via settings or constraints
... discussion we just had leaves me unconvinced we understand how they'll work in the entire process
... applications can talk to eachother
... we have signaling and communications
... side B can say "i want better source video"
... UA doesn't need to do that
... application can
... and hta sent email to list
... there's a pipe metaphor
... stream peer stream
... there's a lot happening on the way
<hta> http://alvesnotes.blogspot.fr/2012/03/byte-stream-fallacy.html
burn: while in peer connection, you should be able to control constraints for sending
... and on other side as well
... streams should be customized for local use
... streams should be customized for peer connection use
... it should have its own set of constraints
... which may succeed/fail
... don't make app do everything
... I'm not at all convinced it'll work right
<Zakim> Josh_Soref, you wanted to ask if browser B has a constraint and to ask if browser B has a constraint and browser A changes constraint to violate B's
Josh_Soref: if browser B has a constraint, and browser A changes the constraint
... what happens?
burn: you said that B had a min-width?
hta: start by scaling
... then in the middle we have SDP negotiation failure
... not sure what it says
... but we have an RFC for it
fluffy: really simple
... I demand left handed threads, you demanded right handed
... the stream will fail
... negotiation fails
burn: same as if user unplugs camera
hta: Ended event
stefanh: ended on track
matthew: I think using offer answer SDP
... will never work fully
... you offer, he answers, but things never match up
... eventually someone rejects
... seems insanely complicated
... let's do it!
... there's a rejected proposal which contains a solution
fluffy: there's comment 22
martin: who wrote it?
matthew: person to my right, ekr [you]
<martin> I wrote it, it's not a big deal
<ekr> I am ekr
<ekr> that is martin
<fluffy> when will comment 22 end
<fluffy> ?
<ekr> the guy in the hat is ekr
<martin> poor Josh
[ scribe note: they totally messed up that transcript ]
<ekr> the guy in the red stripy shirt is martin
matthew: user changes sizes
... see if you can even get SDP that converge
juberti: I think it's possible
burn: I wish we had 3 hours
... half hour for 6 major topics isn't enough
... is there anyone who can hazard what we agreed
<matthew> existing systems that do it use the media path to do it
fluffy: can't repeat anant 's proposal
... I think I agree w/ a fair amount of it
<matthew> and limit the changes to a few codec-related parameters
fluffy: how does this cross remotes
... I think there are a fair number who want remote to change res/framerate
... I would guess those two things would have widespread agreement for wanting
juberti: about remote changes
... they're suggestions
... mandatory isn't going to lead to good things
fluffy: +1
<matthew> consider the case of dealing with negotiating landscape vs portrait when both users have devices that can rotate
burn: optional=hint
Travis: is there widespread agreement on approach of track types mapping to devices?
hta: significant support for that
burn: Video, Audio, Picture?
Travis: that concept
<matthew> nobody sends an offer to send the video in the other orientation and waits for an answer before rotating it/changing the aspect ratio
fluffy: I'm comfortable with that
<matthew> and that's for a very good reason
fluffy: as long as new types of media that aren't video/audio are possible
burn: +1 for that
<martin> the oo purist in me would like to remind that group that prefer composition over inheritance is a good rule
Jim_Barnett: isn't that already today
burn: no, we need a future track for something else
fluffy: find one construct that works for both things
burn: fair number of people who'd like changes for res/framerate
... on sink, possibly resulting in auto changes on source
... hard to argue w/ people who want that
... I'm concerned it'd really work
... not objecting
fluffy: don't want to say "such as"
... say res + framerate -- only
... maybe not possible
... play around, might be able to make it possible
burn: constrain set sufficiently
anant: if you only make a request
... why doesn't the app send a message to itself
... instead of SDP
hta: can RTCWeb have agenda time for it next week
fluffy: no
... we'll still be discussing it next summer
... to anant 's question
... 3 layers
... out of band (xhr)
... sdp
... with several codec specific, and a draft from hta
... and people who say it happens in rtc-rtp
... perhaps if we knew which layer was doing it
anant: I think it's easiest in the highest
fluffy: slowest too
anant: true
fluffy: progress is being made
... hta is writing drafts
burn: in order to get the flexibility for apps
... keep adding layers at lower level
... which has to reflect at other layers
... then eventually you do it at the higher level
matthew: other reason is if you really like SDP
Travis: when you apply a constraint via getUserMedia to a device
<matthew> (which we must here, otherwise we'd have made another decision some months ago)
Travis: does group have agreement that that defines the valid range of settings at any time after
... or not
... do you have to turn off app/turn back on to reset?
hta: if we model change of constraints
... then you can logically change limits
... but if you move outside device support, then setting fails
... on list, we discussed
... if browser UA decides to change params
... without consulting w/ app
... then it should only change w/in current constraints
stefanh: we don't know what to do if it must move outside
hta: mac cameras drop framerate
Travis: anant and I sync up on changing settings w/ constraints
dom: originally getUserMedia was near LC
burn: not this week
dom: doesn't need well defined
... just media stream hierarchy
fluffy: I'd guess a year
dom: you create one document w/ what's stable
burn: 5 months, optimistic
fluffy: I'll go for that, very optimistic
martin: you said "automatic"
... you set constraints on receiving end
... that concerns me
... you said automatically send to sender
burn: that's what I'm understanding
... may magically
... things propogate magically
Josh_Soref: recieving side wants big frames NOW, don't wait for remote stream.
Jim_Barnett: you have a fair amount of control by doing things in <video>
... and say everything else is done to apply across the wire
matthew: if we don't have your draft
... then you could do it out of band (not choice)
[ Constraints / runtime / coffee time ]
burn: I'll run through, no questions
[ hta's video proposal constraints - IETF ]
[ Travis's proposal - device settings ]
Travis: dimension = width+height at the same time
[ gmandyam 's proposal picture constraint ]
burn: thanks Josh_Soref for scribing
hta: we want to open the speaker queue
... people have had a few minutes to think about what we talked about before the break
<martin> +1
hta: it was suggested that we have a five day interim and not let people out before we have decided
... (joke)
... are there question or comments on this topic?
(topic is: what constraints/setting- do we support)
fluffy: there are a lot of things to set on a camera
hta: there could be proprietary ones
fluffy: what do user understand when it comes to settings?
Travis: get v1 out the door
... with a minimal set
<fluffy> hi
<adambe> test
burn: Was strongly encouraged to name it for ownership by one group.
<adambe> (scribe lost connection)
<martin> gmandyam: not keen on going to two bodies to get values added
fluffy: it's not a IETF specific registry (constraints)
burn: years ago.. IEFT and W3C didn't work together on registries
fluffy: any organization can modify the registry
dom: if we want interop we need to test the constraints
gmandyam: v2 of document may be fiction
fluffy: burn, what's the process for the registry
hta: I think it was "expert review"
burn: the entries are registered in the registry, but they are defined in the w3c spec
gmandyam: living document is a red herring
... I think we need to pick a set of constraints for v1
hta: I don't think you can modify constraints
gmandyam: I meant to modify the list
<fluffy> The current registry is Specification Required with Expert Review" registration policy as defined in RFC5226 [RFC5226].
Jim_Barnett: if a constraints isn't documented in w3c, where is it registered?
burn: there's an IANA section in both documents
<fluffy> The registry creation document is at http://tools.ietf.org/html/draft-burnett-rtcweb-constraints-registry-02
Jim_Barnett: so between v1 and v2 you couldn't add constraints?
burn: a w3c document specifies which constraints from the registry that needs to be implemented to be compliant with the spec
<fluffy> The other relevant spec is http://tools.ietf.org/html/rfc5226
burn: the registry should prevent organizations from registering general names that we might want to have for "official" use later
<martin> http://dvcs.w3.org/hg/dap/raw-file/tip/media-stream-capture/proposals/SettingsAPI_proposal_v4.html
<martin> the above contains a proposal for the initial set of constraints; provide comments on this
anant: where was a question about which constraints we should support for gUM
... we should have one list
<martin> gmandyam: you would be able to attempt to set constraints, but these settings would fail
(scribe got thrown out)
discussion about mic input level
juberti: we could skip mic input level in v1
... AGC solves a lot of cases
fluffy: I agree with juberti
burn agrees with juberti as well
[ v5 that anant and travis agree on will be sent out ]
<dom> ScribeNick: Josh_Soref
hta: does anyone remember the structure of errors?
... a number plus string
anant: it should be a string
hta: I got a question from my impl team
... they want more failures beyond permission denied
... like no camera
juberti: no camera, failed, permission denied
Travis: constraints not happy
juberti: hardware not happy
<burn> FYI, media constraints registry policy is "Specification Required with Expert Review"
anant: current getUserMedia has only one error
... it should have more
... for constraint API, it should have a single error kind
hta: no camera, camera malfunction
fluffy: it's about who gets the support call
hta: objection to adding these three codes?
Jim_Barnett: if t1 and t2 are in stream 1, they're sync in that context
... but if t1 is in s 2 and t2 is in s3, they aren't synced?
derf: they're probably sync'd in our current impl
... t2 in a second video window may be delayed
Jim_Barnett: and sync'd w/ different audio if they have different starting points
... derf no container format allows adding and removing of tracks while providing stream encoding
... there was a request to add/drop streams as you go
fluffy: this makes sense
... I've wondered why
... why do we deal w/ container format
... inside the browser
... why don't we allow the browser to construct the container format
<martin> I propose that we rename MediaStreamTrack to MediaStream and MediaStream to MediaStreamSync
fluffy: I'm worried about the topic of choosing a container
Jim_Barnett: if you want a recording, it needs to be in a container
... so it can be played back
fluffy: the browser needs to provide the raw data
... and someone needs to be able to save it to disk
anant: what do you mean raw data?
... pcm?
matthew: flv?
fluffy: you know I have an answer
... I'd probably go w/ a compressed version of the data
derf: w/ compressed data, you need timestamps
matthew: flv
derf: backreferences
martin: you end up w/ a container format
Jim_Barnett: it's a requirement that the base case is simple
... so it should be an appropriate container
fluffy: I know how to solve it
... but all of the containers are workable
derf: that's mostly naive
fluffy: not too naive
... if we're going down a path where the browser returns a container format
... it isn't useful unless we specify which container format the browser uses
derf: you have canvas.getDataURL with a mimetype
... the browser passes back data in that mime type
... or I don't support that
fluffy: is there a mandatory to implement set?
hta: reference normatively the media source spec
... and reference whatever they mandatorily implement
fluffy: they don't have one
... they don't intend to get one
Travis: all the better
derf: there's an open bug for that
fluffy: do you believe they intend to fix it?
... the point of recording is to be able to do something w/ the data
Jim_Barnett: as long as the browser can play it back?
Josh_Soref: if you can take the data and have the browser reproduce it as a stream source
... which can be sent to <video> (with extract via draw to canvas / and imaginary audio API),
... or sending to a peer stream
fluffy: I agree with you
... there should be an API to get raw access to the data
... I don't think it should be this API
hta: media source API has a goal to allow byte stream access
[ hta is reading the spec requirements ]
fluffy: the draft does what I suggested
... which is the opposite of what they said in the room
Travis: are you recording to MPEG4 or akin to that
fluffy: something like that
... I suspect half vendors are thinking one open source format
... unlikely to be supported by the other vendors
... making it largely useless to most people
Travis: having only one browser in the world would make everyone's life easier
fluffy: have raw audio samples
... can wrap in container A or B
... wrapping can be done by JS
... why not just give the raw audio samples?
Jim_Barnett: how do you sync?
anant: I agree on PCM
... but what do you do w/ raw video?
martin: Frame, Frame, Frame
... how do you get pixels off canvas?
anant: can you implement this in js?
Josh_Soref: sure
fluffy: there's a WebEx does H264 in JS
[ ekr has a 1x1 codec ]
[ laughter ]
fluffy: you wouldn't want to re-encode video if you had them that way
derf: I'd rather give you the
fluffy: if you gave me the data and RTP time packets
... that's all I need
... I wouldn't really want that API, but it's provably possible
derf: I agree it's possible
anant: if you get the data over peer connection
... we've agreed on the MTI codec
... if we can agree on the codec
fluffy: sure
anant: peer connection between IE and Firefox w/ VP8 or H264
... then the container isn't a big deal
dom: I'd like a recording API today
anant: if we don't have MPEG today
... I'd like an API that lets the app ask for a format, and try
... that's how the web works today
<martin> no one wants my MTI codec : http://tools.ietf.org/html/rfc2435
derf: I think that's how the API should work regardless of codec or container
Travis: I agree w/ derf+anant
fluffy: ok
[ Recording ]
Jim_Barnett: sync'd recording at media stream
... not at track level
... and people want to add media stream as you go
adambe: you could squeeze in media stream constructor
Jim_Barnett: got a media stream to a <video>
... and you want to record what the user is seeing
<matthew> I suggested MJPEG yesterday, didn't I?
adambe: to record what you're seeing in <video>
... you could record that
Jim_Barnett: you're going to want to use a media stream you're already using
martin: if you have stream, then you can do track by wrapping it in a stream of just that
Jim_Barnett: yes, but not an API for tracks directly
... encoding function for stream at interval
... another for whole blob
... stream encoding must ignore added tracks
... per derf, no container format supports this
... remove a track, provide silence/black
... probably have to fake fourth track
Josh_Soref: couldn't you cheat and always have 15 dummy tracks?
Jim_Barnett: there's no way to do that today
... that'd make it work
matthew: multiple things raised in the last 45s
... you can't add tracks
... and you can't do anything to them?
derf: changing res/frame rate is generally not a problem
... some formats don't support it
matthew: what about dummies
Jim_Barnett: suggestion was dummy tracks
matthew: we need dummy tracks anyway
Jim_Barnett: I didn't think we had them
matthew: it's supposed to be on the todo list
Jim_Barnett: we could say you can do anything the ua can do
... but if we know, we should state up front
... it's always possible to fail
... changing res
... if that fails in some format
derf: I can make it work, by rescaling to dest res
matthew: or just make everything one pixel
fluffy: it looks like advanced authoring format does allow it
... more accurate statement is
... "the one format we want to use will support it doesn't support it"
matthew: mp4
... it's true if you write it up front
... but not at the back end
derf: I don't think things will play it back correctly
matthew: that's different
derf: they won't play
fluffy: what if you write it at front and back patch
derf: if I know up front, I can
[ cross between fluffy and derf about supported ]
fluffy: windows players will support [something]
derf: I'd be shocked
Travis: does anyone object to the general ability to
matthew: mpeg2 transport stream definitely does
derf: I challenge you to find something that reliably supports it
... there's one format people hate more than ogg
... it's mpeg2 transport streams
Travis: does anyone object to specifying limits that you can't add/remove when you're recording under stream conditions
matthew: I think you should get an error when the stream format doesn't support X
... you could support a container format that doesn't support aspect ratio change
... and the user changes
adambe: if I record, and the aspect ratio change, where is the error?
... I guess from the recorder
Jim_Barnett: the stream is valid
adambe: so the recorder needs to throw errors
derf: when you start recording, it records the tracks you had then
... and ignores others
matthew: why not allow it
... and let others do it in the future
fluffy: I like matthew 's phrasing
... an error from the recorder when it doesn't support x
martin: recording
... stopping when you get an error - sucks
matthew: you could have soft errors
... when you flip h/v
fluffy: question is stop or not
Jim_Barnett: streaming
... you return as much data as possible
... return error
... return new stuff as new format
derf: there's a way to do that
matthew: mpeg2 ts is like that
derf: if you stop all streams, and start recording a new set of streams
... that works fine in ogg
Jim_Barnett: streaming recording
... you / someone produces a change that produces unhandleable
... recorder produces an error
Josh_Soref: it should stop the data, send error, and unless you do preventDefault() it should provide new recording automatically
derf: the larger error is the data is flowing async through your stream
... that you haven't seen
... if you start recording some time in the future
... anything in the future
anant: don't interrupt what was happening unless you absolutely have to
... the callback should have the option to let things continue
adambe: I think a specific API is better than that
anant: I'd say return true/false
Travis: that means we define these error events as being Cancelable
... with a specific behavior
martin: cancelable happens
... what happens if you cancel
hta: make sure next ver of proposal is clear about what happens
matthew: for orientation constraint
... can you prevent that from happening
Travis: my view differs from hta's
... and I'm slowly becoming a convert
... scenario
Josh_Soref: Scribe wants to record data coming in from peer connection
... and the peer connection input changed orientation
Travis: tracks have possible incompatible changes
... that may/may not affect recording
Jim_Barnett: blob recording
... we need to be clear about if you're asking for a Blob or asking for Streaming
Travis: the blob container
... to overload the term
... blobs can be anywhere
<matthew> you flip your camera... the offer comes to me with the new aspect ratio... I send answer accepting the new aspect, but in the meantime my user has rotated their screen to see your video correctly... your user now sees rotated video, so they flip their screen again, only they can't send an offer because their offer is outstanding. meanwhile, my side sends an offer (it can, because an answer was already sent)... so when your side goes to send an of[CUT]
<matthew> now offer-offer glare.
[ Open Questions ]
martin: I like the property to prohibit multiple concurrent recordings
<matthew> if your users keep doing this for 11 hours, I can fly back to the US from here to sooner than the O/A exchange converges
martin: as an implementation detail that would fail
Jim_Barnett: there's a UC for user is talking
... and you want to record to file and do speech transcription concurrently
hta: resource constraint
... makes sense to say starting a recording can fail
Jim_Barnett: oh yes
... you can get a failure at any time
hta: no need for a specific restriction
Jim_Barnett: not a specific constraint in the recorder
martin: there may be limitations in your implementation
Josh_Soref: these edge cases are nice to talk about here
... but if they're not that common it's likely that developers won't handle them
... random failure is the only way to get developers to handle these kind of rare errors
... there are no good solutions
martin: we could enable developers to trigger these them selves
Travis: it'd help if we talked in the document about how to trigger these errors
... so that if you're reading this, you'll know that you need the error handler
Jim_Barnett: I think from this, there's enough to write something
... is this a new document?
hta: from dom, it seems like a separate document
... for the time being
adambe: even streaming recording is using blobs
Josh_Soref: "at once recording"
... "chunked recording"
gmandyam: is this a v1 v2 feature?
Jim_Barnett: as a separate document, it isn't before v1
<matthew> "at most once"
gmandyam: simple media stream recording could be added to v1 and more advanced recording could wait to v2
... media recorder in android and other mobile devices is pretty restricted
... we could do something similar in v1
... (i.e. simple version)
Travis: do you mean that this proposal would be v2 then?
<dom> scribenick: adambe
gmandyam: yes
hta: I think the described solution is pretty simple
fluffy: I'm interested in access to uncompressed data
(that was a question to derf)
derf: we had an idea about a MediaStream processing API
... but then we went for the Web Audio API
... which doesn't handle video
stefanh: I think we're done with recording
... next item is identity tagged media streams
gmandyam: I wanted that discussion to take place in the MC TF
ekr: I'll make sure that the comments are forwarded to both lists
stefanh: let's move on
... next topic - generating fake tracks
fluffy: I want to create an object similar to what you get from gUM
... it might be a 1x1 pixel video
(needs license from ekr)
matthew: I want to attach video to a PeerConnection so it's included in the negotiation
... when I have real access to devices - I swap out the dummy
... for audio, if we don't fix this we will have a lost "hello"
... people will click OK and say hello
... there's no transport ready
... I don't care how we solve this
... silent audio, black video
derf: you could use a processed media stream
... you could feed the output from a video tag to it
matthew: it's too complicated for my case
gmandyam: you could generate comform noise
derf: it's similar
matthew: this could maybe work
... soft phones will suck if you can't provide fake streams (which can be swapped for the real thing)
stefanh: I think we need this
... someone should draft an API for it
fluffy: how do I take an web audio object and create a media stream from it
... ?
derf: I don't know if it's in the spec (scribe missed some stuff here)
fluffy: if the default constructor created a empty silent stream
... we should define such an object
martin: a problem is that we can't connect that to the peerIdentity solution
gmandyam: so the problem is that the app is not trusted?
(several people); yes
Travis: is the purpose for fake streams a promise for a stream in the future
matthew: it has to look like stream from a camera and a mic
... I don't want to renegotiate
hta: so you wan't to send the new stream on the same m-line with same SSRC
matthew: the problem is that the session is receive only for the party being called
... how do I respond to a rend-recv if I don't have anything to add?
fluffy, ekr: could this be solved with pr-answer?
matthew: no
[ martin draws on board ]
[ cullen is drawing ]
[ discussion: if the A side receives data after it has offered media to the B side. Can it do something useful with it -- before getting the answer? ]
[ white ink on white board :) ]
<fluffy> @dom, I think we are getting closer back to w3c land
hta: trying to summarize
… we are trying to get the information to construct an answer for the SSRC for the video that will be added later when the user consents
… suggestion is to have getUserMedia immediately return something that can be added to PeerConnection before there is consent
stefanh: we have no way for caller to reject some of the streams the callee wants to send
juberti: clue folks are looking at 4 way handshake for this
… clue people are still working on this
anant: the two objections to make gUM sync: 1) breaks existing applications 2) Imposes bad user experience
<martin> I'll summarize the proposal: getUserMedia immediately returns a MediaStream with the requested tracks. These tracks are fake until consent is acquired or consent is denied (which causes the track to end).
ekr: people will set the "promise object" to a video src directly without using the success/error events
richt: You're proposing something Promisee-like that isn't Promise compliant. Promises require .then callbacks with a success and error callback. So if we do this we do that. And we already have that already (minus the sync return object).
... i.e. we already have success and error callbacks, just not wrapped in .then.
ekr: the existing callback forces the developer to wait until a valid stream is returned before using it
... the new proposal doesn't enforce this
<richt> so it's either mediaStream = getUserMedia(opts, promiseSuccess, promiseError) or mediaStream = getUserMedia(opts).then(promiseSuccess, promiseError) ?
anant: the result from gUM doesn't need to be a "real" MediaStream
juberti: I don't like that this break existing code
... a new suggestion: let gUM return an object and fire the callback as well
... then we wouldn't break existing code
<richt> +1 to jubertI's proposal as I explained above.
burn: we're not in last call
... we will break code up until we get to last call
hta: I propose a gUM2()
martin: two ways to do something is really bad
hta: I think we understand the problem
stefanh: we have two problems
martin: num_problems > 1
... is true
end of meeting!
puh!