RE: Another image capture API proposal (Re: Fwd: Re: Re: Proposed resolution for open items)

Randall,
  Could you explain what you mean by the  'use-once aspect'?  As I read
the file api document, the URL created by createObjectURL is valid until
it is explicitly revoked (or the page is destroyed).  I understand that
direct access to (and assignment of) the underlying stream can be
convenient, but is it necessary for your example to work?  

- Jim Barnett  

-----Original Message-----
From: Randell Jesup [mailto:randell-ietf@jesup.org] 
Sent: Tuesday, May 15, 2012 9:30 AM
To: public-media-capture@w3.org
Subject: Re: Another image capture API proposal (Re: Fwd: Re: Re:
Proposed resolution for open items)

On 5/15/2012 6:46 AM, Harald Alvestrand wrote:
> Changing the subject line in order to make this thread findable....
>> This is about an app grabbing a preview stream and then, when the 
>> user presses a "take photo" button, firing the flash and grabbing a 
>> high-res snapshot, right? Here's a thought experiment:
>>
>> <video id="preview" onclick="takePhoto(event)"></video>
>> <script>
>> function onsuccess(stream) {
>>   document.getElementById("preview").src = stream; } function 
>> blobCallback(blob) {
>>   // This blob contains the image data; do whatever needs to be done 
>> with it } function takePhoto(event) {
>>   var stream = event.target.src;
>>   var videoTrack = stream.videoTracks[0];
>>   videoTrack.takeSnapshot({}, blobCallback); } 
>> navigator.getUserMedia({video:true}, onsuccess); </script>
>>
>> Apart from allowing HTMLMediaElement.src to be a MediaStream (already

>> on mozilla-central!),

> Actually this shows that HTMLMediaElement.src can be fetched, not just

> written to.. I hadn't realized that part of the proposal.

Actually, that's something that was basically TBD.  And we might need to
think if this is the best way to do it, and if it might cause any
problems for other things that might read src - we may need to have a
different way (.stream, .src_stream) to retrieve the original stream
pointer, and return some type of string for .src (right now I think
mozilla's implementation returns "").  With a createObjectURL in .src,
we might be able to do something similar if we add new getter
properties.  I don't think we can just return the original string and
then use it in a hypothetical URL.objectFromURL() because of the
use-once aspect.

The last option (for either .src = stream or .src =
URL.createObjectURL(stream)) is to have an implicit
URL.createObjectURL() when you retrieve .src if it refers to a
MediaStream.  For .src = stream this works (though you can't control the
one-use flag); for .src = createObjectUrl it also has an odd property
that video.src = x; y = video.src yields x != y.

I think either y = video.src (if that doesn't mess up things that want a
string returned by .src) or y = video.stream (or .src_stream) otherwise.

  video.src_stream might be less confusing, if you want to be able to
also get the output of the video element in
video.captureStreamUntilEnded() per roc's MediaStream Processing
proposal.

>> the only new API here a VideoMediaStreamTrack subclass of 
>> MediaStreamTrack, with a takeSnapshot method on it. I've given that 
>> method a blob callback to asynchronously return the snapshot image, 
>> and an options object. By default I think the snapshot should 
>> autofocus, autoflash, etc, but you could provide options to override 
>> all of those things. I'm not enough of a domain expert to say what 
>> all those options should be. By default, takeSnapshot should obtain 
>> the best quality and highest-resolution image the hardware can
provide.
>
> Like the SendDTMF function on the LocalAudioStream (not yet sure how 
> to specify all this subclassing in WebIDL), this seems reasonably 
> natural to me.

Regardless of how you get ahold of the track, this part of the proposal
does seem pretty natural and clean once you're dealing with
MediaStreams.  It does require *some* method of retrieving the track
from the video element.

> A quick implementation would copy out the framebuffer (if there is 
> such a thing in the implementation); a more advanced implementation 
> could do what's written above.
>
> I do miss an option to set the format of the image, but I assume that 
> is either a natural format for the platform (the same thing as being 
> picked out of a Canvas?) or an option (constraint?) on the
takeSnapshot call.

Yes, I think that's the point of the {} in the call.


--
Randell Jesup
randell-ietf@jesup.org

Received on Thursday, 17 May 2012 17:59:07 UTC