RE: [MSE] Issue on seeking from the end time position

Hi Aaron,

Here is a sample code which was modified from the section 13. “Examples” in the MSE spec. (The red colored texts are where I modified.).

This sample supports the seeking when the readyState of MediaSource becomes “ended” by endOfStream().  As you said, the issue is about when appendBuffer() is reopened. I raised this issue since I am not sure that the following behavior of appendBuffer() at seeking is right. If right, it would be helpful for developers to include the following (red colored) NOTE in the section 2.4.3 Seeking. Alternatively, the spec should be modified as that Seeking changes the readyState from “ended” to  “open” automatically. The modification will make the existing “Examples” work well.

  function appendNextMediaSegment(mediaSource) {
   if (mediaSource.readyState == "closed”)
      return;

    // If we have run out of stream data, then signal end of stream.
    if (!HaveMoreMediaSegments()) {
      mediaSource.endOfStream();
      return;
    }

    // Make sure the previous append is not still pending.
    if (mediaSourcereadyState == “open” && mediaSource.sourceBuffers[0].updating)
        return;

    var mediaSegment = GetNextMediaSegment();

    if (!mediaSegment) {
      // Error fetching the next media segment.
      mediaSource.endOfStream("network");
      return;
    }

    mediaSource.sourceBuffers[0].appendBuffer(mediaSegment);
// NOTE  if readyState == “ended”, the sourceBuffer will be reopen. the web application should handle the “sourceopen” event.
  }

  function onSeeking(mediaSource, e) {
    var video = e.target;

    if (mediaSource.readyState == "open") {
               // Abort current segment append.
              mediaSource.sourceBuffers[0].abort();
}

    // Notify the media segment loading code to start fetching data at the
    // new playback position.
    SeekToMediaSegmentAt(video.currentTime);

    // Append a media segment from the new playback position.
    appendNextMediaSegment(mediaSource);
  }


-***---***---***---***---***---***---***---***---***--***---***---***-
Tatsuya Igarashi (Tatsuya.Igarashi@jp.sony.com<mailto:Tatsuya.Igarashi@jp.sony.com>)
Information Technology Development Div, R&D Platform
Sony Corporation


From: Aaron Colwell [mailto:acolwell@google.com]
Sent: Friday, September 06, 2013 12:28 AM
To: Igarashi, Tatsuya
Cc: public-html-media@w3.org
Subject: Re: [MSE] Issue on seeking from the end time position

Hi Tatsuya,

I think we still may be misunderstanding eachother. It would be really helpful if you could post minimal concrete HTML & JavaScript that exposes the issue you are seeing. That would make it much easier for implementers to figure out what is going on.

Comments inline.

On Thu, Sep 5, 2013 at 4:59 AM, Igarashi, Tatsuya <Tatsuya.Igarashi@jp.sony.com<mailto:Tatsuya.Igarashi@jp.sony.com>> wrote:
Hi Aaron,

Since I saw this issue in both Chrome version 28.9.1500.95 and IE11 version 11.0.9431.181, it would not be a bug in the MSE implementations.

There have been some duration related fixes since Chrome 28. Please retry with a beta or dev channel build to see the issue you are encountering has been fixed.


I think the seeking behavior is affected when the readyState became “ended” by endOfStream(null).

Seeking behavior should not be effected by the readyState becoming ended. The only slight difference might be caused by the seekable range changing  because of how the buffered ranges & duration change on the transition to "ended". endOfStream() triggers the duration change algorithm, which updates the duration. The "ended" state also changes how the buffered attributes behave so a transition back to "open" may reveal that there is now missing data that needs to be appended before the seek can be completed.

The both implementations , i.e. the append() and the appendBuffer(), follows the spec as described in “3.5.4 prepare append algorithm”.  Would you confirm that the sourceBuffer is reopen by the methods and the “source open” event is fired in the case of seeking ?

I think you are mixing behaviors. append() & appendBuffer() should trigger a transition from "ended" to "open" if they are called. This transition is not related to seeking. It is solely a result of calling append() or appendBuffer(). I think you might be confusing these behaviors because you are appending from a "seeking" event handler.

One thing to note about the transition to "open" is that if you seek to a value really close to the end, the seek may not complete if you don't call endOfStream() because there might not be enough data for the media engine to complete its preroll. Only after calling endOfStream() again does the media engine know that it has all the data and can complete the seek. This should only be a problem if you seek within 100-150ms of the end of the stream on Chrome.


If it is a right behavior, I can manage to address my issue. Also, it would be nice if there is a NOTE at “2.4.3 Seeking”

It is still not clear to me whether a note is really needed. Can you please provide some suggested text if you still think it is necessary.


In the case that endOfStream() is not invoked but video.currentTime becomes equal to video.duration, the video.ended became true but the event of “ended” was not fired. I think that this is a bug of implementation. Right?  I will file the bug.

Yes there is a bug, but it may not be what you expect. Since endOfStream() has not been called, the HTMLMediaElement shouldn't actually be setting the ended attribute to true because it doesn't actually know that the true end of the presentation has been reached. The bug, in my opinion, is that ended should never become true unless MediaSource.readyState is "ended" AND HTMLMediaElement.currentTime == HTMLMediaElement.duration. This needs clarification in Section 7 of the MSE spec. I'll file a bug to make sure text is added to cover this behavior.

Aaron

vide
Thank you.

-***---***---***---***---***---***---***---***---***--***---***---***-
Tatsuya Igarashi (Tatsuya.Igarashi@jp.sony.com<mailto:Tatsuya.Igarashi@jp.sony.com>)
Information Technology Development Div, R&D Platform
Sony Corporation




From: Aaron Colwell [mailto:acolwell@google.com<mailto:acolwell@google.com>]
Sent: Wednesday, September 04, 2013 12:23 AM

To: Igarashi, Tatsuya
Cc: public-html-media@w3.org<mailto:public-html-media@w3.org>
Subject: Re: [MSE] Issue on seeking from the end time position

Hi,

This sounds like a bug in the MSE implementation. I'm assuming you are seeing this in Chrome, so please file a bug at http://crbug.com/ . Please include the version of Chrome, a simple page that reproduces the problem, and assign the Cr-Internals-Media-Source label to the bug.

Further discussion of this particular issue should probably not happen on this list. The seeking behavior should not be effected by where the seek was initiated from. The steps in the spec outline what the proper behavior should be, if Chrome isn't following the spec in this case, then we'll fix the implementation. Thanks for bringing the issue to my attention.

Aaron

On Sun, Sep 1, 2013 at 10:30 PM, Igarashi, Tatsuya <Tatsuya.Igarashi@jp.sony.com<mailto:Tatsuya.Igarashi@jp.sony.com>> wrote:
Thanks, Aaron.

The problem which I observed on seeking from the end time position is as follows;


1.     On seek event,  the web application invokes appendBuffer() to add the data at the new position.

2.     The prepare append algorithm described 3.5.4 runs. At step 3, the sub-steps reopen the source Buffer because the readyState of the mediaSource is “ended” by the endOfStream(null).

3.     The event of “sourceopen” is fired  and it seems to clear the currentTime state of the media element object and it cannot resume the seek algorithm as normal seeking.

I tried to resolve this issue by storing the currentTime on seek event and the web application resumes the seeking by itself.  But I think that it is better if the normal seeking steps described in 2.3.4 can run even in the case of the endOfStream(null) is invoked. So, I wonder that seeking from the end time position should change the currentState from “ended” to “open”.

You said that the transition from “ended” to “open” should only happen if the application appends more data. Does the transition described in the step 3 of the prepare append algorithm ?  Does the transition assume this case of seeking from the end time position ?

Thank you.

-***---***---***---***---***---***---***---***---***--***---***---***-
Tatsuya Igarashi (Tatsuya.Igarashi@jp.sony.com<mailto:Tatsuya.Igarashi@jp.sony.com>)
Information Technology Development Div, R&D Platform
Sony Corporation



From: Aaron Colwell [mailto:acolwell@google.com<mailto:acolwell@google.com>]
Sent: Friday, August 30, 2013 11:41 PM

To: Igarashi, Tatsuya
Cc: public-html-media@w3.org<mailto:public-html-media@w3.org>
Subject: Re: [MSE] Issue on seeking from the end time position

I'm not sure I understand the problem you are having. The MediaSource.readyState being "ended" should not effect seekability. You should be able to seek whether the readyState is "open" or "ended". Why do you think there needs to be a transition from "ended" to "open"? I think this transition should only happen if the application appends more data, which is what the current spec says should happen.

Please provide more details so I can better understand the issue you are having.

Aaron

On Fri, Aug 30, 2013 at 2:33 AM, Igarashi, Tatsuya <Tatsuya.Igarashi@jp.sony.com<mailto:Tatsuya.Igarashi@jp.sony.com>> wrote:
Hi,

I have a problem in the case of seeking from the end time position, i.e. video.currentTime== video.duration. The following NOTE of EndOfStream() descibles a very useful idea for the MPEG DASH and I use the endOfStream(null) when all media segments are appended to the source buffer. However this way does not allow to use the existing source buffer as the seek algorithm since the ready state becomes from “open” to “ended” by the endOFStream(null).  I wonder that seeking from the end time position should automatically change the readyState of the sourceBuffer from “ended” to “open” if endOFStream() is called without an error.

NOTE
This allows the duration to properly reflect the end of the appended media segments. For example, if the duration was explicitly set to 10 seconds and only media segments for 0 to 5 seconds were appended before endOfStream() was called, then the duration will get updated to 5 seconds.

Thank you.

-***---***---***---***---***---***---***---***---***--***---***---***-
Tatsuya Igarashi (Tatsuya.Igarashi@jp.sony.com<mailto:Tatsuya.Igarashi@jp.sony.com>)
Information Technology Development Div, R&D Platform
Sony Corporation