Re: Proposal to update the spec

Hi Francois and CG members.

Thank you Francois for your excellent work and recent proposal. We very
much share your opinion that CG specifications must detail how the timing
object

   1.

   connects to online timing objects
   2.

   connects to media elements
   3.

   connects to timed data (e.g. using TextTrack objects).


Please find our thoughts on your proposal below.

Proposal 1)

Timing objects connect online timing objects through a (client-side)
TimingProvider interface. This implies that

   -

   timing providers may readily supply custom implementations of the timing
   provider interface through JavaScript libraries.
   -

   browser vendors may provide a built-in Timing Provider for potential
   future specification of timing protocol (communication between online
   timing service and user agent). Such protocols may also exploit lower level
   communication mechanisms such as UDP.
   -

   Timing providers could use browser plugins to provide access using other
   communication mechanisms, such as UDP.
   -

   Support for local timing objects may be provided by web browsers (or
   libraries) simply by implementing a specific LocalTimingProvider object.


This is very much in line with our intuition.

Proposal 2)

Timing objects connect with media element through a “timing” attribute. The
use of the “timing” attribute indicates “slave mode” whereas no “timing”
attribute indicates the default “master” mode. “Slave” mode implies
playback follows the timing object no matter what, even if that implies
skipping media content. “Master” mode implies stopping playback when
buffering is required, as we are currently used to.

Again. We fully agree with this proposal. Maybe even name it “timingsrc” to
emphasize that the attribute identify an external timing resource.

Proposal 3)

Timing objects are connected to timed data by integrating directly with
existing TextTrack objects. Essentially, you should be able to get a
TextTrack object from a TimingObject just the way you currently would get
one from a MediaElement.

We have some reservations with this proposal. In the bigger picture, we
seek loose coupling between timing objects and time-sensitive components.
In proposal 2 we see this pattern clearly as media elements explicitly
takes direction from an externally referenced timing object. We would
ideally like this to be the case also for the TextTrack integration. So,
instead of adding a .addTextTrack() method to the TimingObject we would
instead like TextTrack objects to be constructed with a “timingsrc”
attribute or similar. This could be achieved in many ways. For example, we
could make a new constructor for the TextTrackObject that takes a
timingobject as parameter, or we could make a TimedTextTrackObject wich
combines timing with a normal TextTrackObject

In any case, loose coupling between the timing object and the sequencing
logic of the TextTrackObject would be beneficial in several respects

   -

   make it possible to dynamically switch timing source for a TextTrack
   during playback. For instance, a social media application might allow video
   UI to indicate the moving playback position of friends who are also
   watching the same video at the same time. Joining them for a collaborative
   session would require switching to a shared timing resource.
   -

   be consistent with how media elements and other components are connected
   to the timing object
   -

   keep the API of the timing object simple (and clean of references to
   consuming components)



Even though this might not be the easiest road in terms of specification,
we think it would still be worth an extra effort to see if it can be done
this way.


Additional Proposal : Chaining

In addition we would like to suggest that timing objects should support
chaining. Chaining would mean that a timing object (child) could take
another timing object (parent) as timing source.

A child timing object would be useful for providing an alternative
representation of the parent timing object. For instance if a parent timing
object represents position/velocity in milliseconds, a child timing object
could represent the same timing source in seconds, or perhaps beats per
second or frame number, depending on what makes sense for a programmer or a
specific timed component.

Another important use case for this would be to “skew” a timing object so
that it matches the timeline of some given media or timed data. For
instance, this strategy would make it easy to play a video between
[200,324], even if the video comes with an internal timeline of [0,124].
The same is true for timed data (e.g. TextTracks). The ability to chain
timing objects generally implies that re-indexing of the media is not
necessary, as it is presumably easier to just skew the timing object.

Chaining may also be used to support dynamic switching of timing objects,
without requiring any changes to the TimingObject API. Chaining only
requires that timing objects allow their reference to parent timing object
to be reset, and that a new update event is generated for children, based
on the new parent. This way, a large set of timed components can easily
switch back and forth between a private and a publicly shared timing object.

Finally, note that chaining also matches proposal 1), where the
HTMLTimingObject essentially takes a TimingProvider object as parent timing
source. Ideally, the TimingProvider interface would at least implement the
TimingObject API.


Best,

Ingar and Njål



2015-07-07 17:26 GMT+02:00 Francois Daoust <fd@w3.org>:

> Hi Multi-Device Timing CG,
>
> Right now, the Timing Object specification is clear, elegant and simple
> but also single-device and not connected to existing elements in HTML5.
> This makes it hard to evaluate its "full potential", especially when
> talking to external people.
>
> I have been thinking on ways to update the spec to:
>
> 1. specify how a timing object can be associated with an online timing
> resource, to fulfill the multi-device promise
> 2. specify how a timing object may control a media element for
> cross-device synchronized video playback
> 3. specify how a timing object may control TextTrack objects to create a
> Sequencer
>
> Here is what I have in mind. I plan to propose spec prose along these
> lines. The result will not be perfect. I left some details out below on
> purpose but even with details in, I expect many open questions to remain.
> However, having some text in place along these lines would allow to reach
> out to potential implementers and gather feedback.
>
> What do you think?
>
>
> 1. Association with an online timing resource
> ==========
> It seems important to leave some freedom for timing providers to control
> the communication and messaging protocols used to interact with the online
> timing service. What this means to me is that there needs to be a way to
> plug an object provided by the timing provider into a timing object.
>
> I propose to:
> - define a TimingProvider interface that timing providers would need to
> implement.
> - add a "timing provider" product to the conformance section since the
> above interface would not be implemented by user agents.
> - add an "srcObject" attribute to the Timing Object interface which, on
> setting, would associate the timing object with the TimingProvider object.
> - adjust the algorithms accordingly
>
> Note a user agent may could provide a native implementation of
> TimingProvider objects. This would be required to support existing
> protocols that are not exposed to Web pages, such as the UDP-based
> communication protocol used in DVB TM-CSS for clock synchronization.
>
>
> 2. How to bind a media element to a timing object
> ==========
> From a media element perspective, a timing object looks like a Media
> Controller with one notable exception: a media element cannot pause a
> timing object when data buffer runs out.
>
> Here, I propose to:
> - extend the MediaElement interface to add a new attribute named "timing"
> that would associate it with a timing object.
> - reference appropriate algorithms, clarifying for instance that the
> "position" of the timing object becomes the "associated clock" of the
> "media timeline" in the generic case.
> - note the need for the user agent to keep up with the position of the
> timing object no matter what, even if that means skipping several seconds.
>
>
> 3. Sequencer
> ==========
> With TextTracks, HTML5 already has a Sequencer in place to some extent.
> The main restriction is that TextTracks are bound to a media element and we
> don't want that restriction.
>
> Instead of creating a dedicated Sequencer interface, I propose to:
> - add an "addTextTrack" method to the TimingObject interface
> - explain that a TimingObject must run the "time marches on" algorithm (or
> a slightly updated version of it) when it is moving, as defined in HTML5.
>
> Reusing TextTracks may not be as beautiful as having a nice Sequencer
> interface but that should be enough to enable the required functionality
> and would require very limited updates to the spec.
>
>
> Thanks,
> Francois.
>
>