Re: captureStreamUntilEnded? (Re: peerIdentity and transparency)

On Fri, Feb 7, 2014 at 9:02 AM, Harald Alvestrand <harald@alvestrand.no>wrote:

>  The only spec for this function I can see is in a draft that isn't being
> pursued at the moment (as far as I know):
>
> http://www.w3.org/2011/audio/drafts/1WD/MediaStream/
>
> The relevant WebIDL is:
>
> partial interface HTMLMediaElement {
>  readonly attribute MediaStream stream;
>
>  MediaStream captureStream();
>  MediaStream captureStreamUntilEnded();
>  attribute boolean captureAudio;
>
>  attribute any src;
> };
>
> We already took the "src" change from this spec and turned it into
> "srcObject", and made it part of our spec. But I haven't seen a proposal to
> move captureStream* in the same way.
>
> Is it interesting to think of adopting the captureStream() interface? It
> would resolve a long standing issue of "how do I produce a MediaStream when
> I don't have a camera", but there might be dragons lying in wait here that
> makes this unadvisable at the present time....
>

The captureAudio and 'stream' attributes are not needed and we haven't
implemented them, so forget those.

I think we should standardize captureStream() and
captureStreamUntilEnded(). I apologize for not pushing it myself, and plead
lack of time. Those methods are exposed in Gecko with a prefix but very
little used at the moment, AFAIK, so we can change things if we need to.

They're mostly not hard to specify, but there are a couple of important
issues:

1) The origin of the MediaStream's data can change over time with either
captureStream() or captureStreamUntilEnded(). With the former, it's
obvious: element.src = "http://foo.com/..." then later element.src = "
http://bar.com/...". The latter is less obvious, but basically loading of a
single media resource can require multiple HTTP fetches of the same URL,
but some of those fetches can be HTTP-redirected to other domains. So, with
these APIs, you need to be able to dynamically mark a MediaStreamTrack as
"private" (or whatever term we're using for tracks whose pixel/audio data
is inaccessible to the page script).

2) <video> playback can block for various reasons --- explicit pause,
buffering due to network delay, etc. Someone has to decide how that affects
consumers of the MediaStream. For example, if you're feeding a
video.captureStream() into a MediaRecorder, you could choose to have the
recorder record silence during the paused interval, or you could choose to
simply elide the paused interval from the recorded stream. There are clear
use-cases for the latter (e.g. video transcoding) and I can't think of any
situation where you would really want the former. Gecko implements the
latter by giving every MediaStream a notion of "blocking intervals".
(Actually, internally we can select those modes per-stream-consumer.)
However, plumbing a concept like that through all MediaStream consumers is
a significant amount of work at the spec and implementation level. I think
we should do it but I do not pretend it's easy.

Rob
-- 
Jtehsauts  tshaei dS,o n" Wohfy  Mdaon  yhoaus  eanuttehrotraiitny  eovni
le atrhtohu gthot sf oirng iyvoeu rs ihnesa.r"t sS?o  Whhei csha iids  teoa
stiheer :p atroa lsyazye,d  'mYaonu,r  "sGients  uapr,e  tfaokreg iyvoeunr,
'm aotr  atnod  sgaoy ,h o'mGee.t"  uTph eann dt hwea lmka'n?  gBoutt  uIp
waanndt  wyeonut  thoo mken.o w