Re: [media] change proposals for issue-152

Many of your objections actually have nothing to do with moving the
element to its own space and can be resolved in the existing way that
we use <track>. Let me elaborate below.

On Sun, Mar 27, 2011 at 11:41 AM, Sean Hayes <Sean.Hayes@microsoft.com> wrote:
> I don't see it as a purity issue. There are 4 main practical reasons that I want to unify the handling of video and text tracks which were present in the version we agreed on at the f2f and were dropped in your CP.
>
> 1 - in a multi-video layout it will be important to be able to place the text tracks so that they encompass and are placed with respect to the whole layout, not simply the video it is slaved to.

This is also possible when we retain the <track> element within the
<video> element - then we have the ability to put captions on the
individual viewports of the slaved videos, too. That would be rather
complicated to create when the captions are separate.


> 2. There does not need to be another special set of rules for downloading the video resource, the same rules and states should work for both.

This is only partially true. The networkStates can indeed work for
<track> and we may still want to harmonize them with the mediaElement
states. However, the readyStates are total overkill for a text file.

  const unsigned short HAVE_NOTHING = 0;
  const unsigned short HAVE_METADATA = 1;
  const unsigned short HAVE_CURRENT_DATA = 2;
  const unsigned short HAVE_FUTURE_DATA = 3;
  const unsigned short HAVE_ENOUGH_DATA = 4;
  readonly attribute unsigned short readyState;


> 3. Track should have the same source selection algorithm as <audio> or <video>.

This is again something that is a criticism of how <track> currently
works and has nothing to do with whether it is inside <video> or not.
I would be curious to have a separate discussion whether we should
extend track to have <source> elements underneath. But I would prefer
this to be a separate change proposal. It is not a reason to pull
<track> out from underneath <video>.


> 4. The solution should allow having visible captions slaved to an audio only resource without having to pretend that the audio is a video, or giving the audio element a visible rectangle.

After having considered this situation and overcome my initial
objections to this situation, I've come to see it actually as a good
thing that you have to put an audio file into a <video> tag to give it
a visual representation, including captions or lyrics etc. If you want
to run your own controls and display and just want to have the
TextTrack API available to your audio resource, you already have that
available to <audio>, since the TextTrack API is part of MediaElement.


> I'm not so much concerned with the use case of a text track that is not slaved to a media timeline, however this would be readily achievable simply using a clock since the timeline of a text track is perfectly well defined.

Your proposal starts with the use of a text track that stands alone.
What would be its visual representation? What the use case?


> Secondly I see no problem defining that, in the absence of styling to the contrary, the act of slaving to a media timeline causes the default rendering rectangle to be that of the video it is slaved to.

The element is stand-alone. So, your default rendering cannot rely on
a <video> element that it is slaved to or on the presence of a
viewport. An element inside the <video> can reply on the presence of a
viewport - a stand-alone element can't.


> I do see a very big problem however with defining the video be a viewport and not simply a containing rectangle and having text track rendering restricted to that rectangle. Which makes proposal (2)

Are you concerned about black bars and the like? If I understand it
correctly, the video viewport is everything that is displayed in the
video rectangle, including the black bars and the text track would
render onto this, not just the part where the actual video is
displayed. The controls are rendered onto the complete viewport, too.


> However my main issue is the fact that I don't believe you have adequately justified moving video and audio secondary tracks out of the master video (#10 'the San Diego solution') and an implicit timeline.  The only justification that I can see is that you can apply style, but to my mind that is readily achieved with
>
> video > track[kind='video'] { ... }
>
> And that avoids having to have to have a bunch of special rules for how to handle the redundant syntactic baggage of slave tracks with controls, poster, etc.

So, are you saying that you still favor the #10 solution that we first
discussed in San Diego?

The main reason for moving away from it is that we realized that we
were re-inventing for audio and video tracks exactly the same
functionality that is already present for audio and video elements.
The overlap and code duplication that would be necessary increasingly
out-weighed the advantage of the "cleaner" mark-up. In addition, for
video tracks, it makes a lot of sense to have multiple tracks
displayed next to each other rather than obstruct each other by trying
to render into the same viewport. An author would be utterly confused
if he defined multiple video tracks, but would only every by default
see a single video track. So, dealing with multiple separate elements
and slaving them to a master seemed a simpler approach.


> So perhaps if we were to go back to that point at 2pm on Sunday where we seemed to have agreement, and the solution that I thought you were going to write up on behalf of the group as a CP, rather than your modified version, then maybe we can move forward from that point; incorporating a version of the track selection API to incorporate in band tracks.


I'm sorry, but it seems to me that you might have mistaken a joke for
agreement. At least I was joking when I said that if we were fully
consequential, we should pull all tracks out from underneath the video
element. There are good reasons why that's not possible for text
tracks and I never for a minute wanted to give the impression that I
actually agreed with that idea. I can't speak for the others present
in the room, but after you had left, certainly nobody seemed to have
subscribed to that position and was pushing for it. We all worked
towards what ended up in proposal 2, even if some of the details of
proposal 2 had to be added later and are now also under scrutiny.

Regards,
Silvia.


>
> Cheers,
> Sean.
>
> -----Original Message-----
> From: public-html-a11y-request@w3.org [mailto:public-html-a11y-request@w3.org] On Behalf Of Silvia Pfeiffer
> Sent: 27 March 2011 19:04
> To: HTML Accessibility Task Force
> Subject: Re: [media] change proposals for issue-152
>
> Dear Media Subgroup,
>
> As per decision by the chairs of the HTML WG, we have until Friday to
> continue discussions and come to an agreement on a common change
> proposal: http://lists.w3.org/Archives/Public/public-html/2011Mar/0614.html
> .
>
> I would recommend to approach this by going one by one through the
> differences of at least the three proposals that have originated from
> this group and either come up with a unified new one, or modify one to
> match all our requirements. The discussions between Ian's proposal and
> ours is one that I would like to see happening afterwards on the main
> list.
>
> Here are the three proposals under discussion:
>
> (1) http://lists.w3.org/Archives/Public/public-html/2011Feb/0363.html
> (Frank's original proposal)
> (2) http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Change_Proposal
> (write-up from the F2F)
> (3) http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Change_Proposal_2
> (Sean's proposal)
>
> Since (2) and (3) are very similar and emerged because we didn't have
> time to finish that discussion at the F2F, I would like to address
> this first.
>
> So, proposal 3 suggests that rather than having the <track> element as
> a dependent element underneath the <video> element, we should make the
> <track> functionality available as a prime element to Web pages under
> the name <cues>.
>
> While I can follow the reasoning for this proposal to stem from the
> idea of having a unified approach to all possible types of content
> that can appear in a multi-track resource, I believe this is actually
> taking the purity reasons a step too far. I have two main issues with
> this approach:
>
> 1. Captions and more generally time-aligned text for videos, are
> always slaved to video content. They do not make sense to exist as an
> entity by themselves. The times in the cues of a cue file only become
> real when attached to a video. A <cues> element on a Web page without
> a video would never display anything because the cues would never
> become active.
>
> 2. There should be a default rendering of cues and it should be
> attached to the video viewport. Rendering in other positions on the
> page should be regarded as custom rendering and the 20% case - easy to
> achieve, but not the common use case. Proposal 3 puts this on its
> head, making rendering in other positions on the page the default and
> rendering on top of the video a problem of the Web page author.
>
> For these reasons mainly, I would refrain from trying to re-define how
> the <track> element works.
>
>
> Best Regards,
> Silvia.
>
>
>
>
>
> On Mon, Mar 21, 2011 at 10:03 PM, Sean Hayes <Sean.Hayes@microsoft.com> wrote:
>> No. we didn't agree on pulling it out. We did however have a unified approach on them all being tracks. I disagree with the move of treating them differently, either they are all tracks or they are all top level elements.
>>
>> I will submit my proposal separately then, I'll also rename <track> to <cues>. I agree that we are not done on this issue.
>>
>> -----Original Message-----
>> From: Silvia Pfeiffer [mailto:silviapfeiffer1@gmail.com]
>> Sent: 22 March 2011 04:53
>> To: Sean Hayes
>> Cc: HTML Accessibility Task Force
>> Subject: Re: [media] change proposals for issue-152
>>
>> Hi Sean,
>>
>> I don't think we had all agreed on pulling <track> out into a
>> stand-alone tag. I certainly disagree with that move.
>>
>> So, given the lack of time, we will certainly have to submit both
>> proposals now and leave it to the chairs to decide how to move on. I
>> don't think we've finished discussions on this topic though and
>> believe that given time we could still converge. But we'll now have to
>> wait and see what the chairs will allow to happen.
>>
>> Cheers,
>> Silvia.
>>
>>
>> On Tue, Mar 22, 2011 at 1:56 PM, Sean Hayes <Sean.Hayes@microsoft.com> wrote:
>>> [4] Is an interesting CP, it does however fail to capture one fundamental premise of the proposal that we had on the table when I left at 2pm. Namely that the handling of text tracks should be as far as possible the same as the handling of a media track. This would imply under this formulation that <track> (meaning a text track) should be promoted to the same level as audio and video, and with the same model, so no it doesn't have my agreement that it is an equivalent to [1]
>>>
>>> I am editing a copy of the page with those changes.
>>>
>>> -----Original Message-----
>>> From: public-html-a11y-request@w3.org [mailto:public-html-a11y-request@w3.org] On Behalf Of Silvia Pfeiffer
>>> Sent: 22 March 2011 00:47
>>> To: HTML Accessibility Task Force
>>> Subject: [media] change proposals for issue-152
>>>
>>> Hi media subgroup,
>>>
>>> A quick status update on where we stand wrt change proposals for issue-152.
>>>
>>> Over the weekend we have worked on a change proposal, which we called
>>> the "San Diego Thought Experiment", see [1].
>>>
>>> It is based on the ideal markup representation of a multitrack media
>>> resource, where external resources are represented in markup as tracks
>>> of a main resource. As we worked through the markup changes, the
>>> IDL/JavaScript API changes, the CSS changes and the rendering
>>> approaches, we realized that the implementation of this ideal
>>> representation would replicate far too many existing codepaths and at
>>> the same time introduce complex layout requirements that would be
>>> unrealistic to expect to be implemented.
>>>
>>> We came to the conclusion that the approach of keeping separate
>>> <video> and <audio> elements around and synchronizing them to a
>>> "master" element (i.e. the "main" resource) would be far easier to
>>> implement and just as powerful. So, we picked up the existing option 6
>>> [2] and continued designing from there to see if that would be
>>> achievable in a simpler manner.
>>>
>>> Note that in the meantime Ian has also submitted a change proposal
>>> which is highly interesting [3].
>>>
>>> Since Eric and I promised to put the change proposal that was the
>>> outcome of the F2F discussions together, we've worked on this today
>>> and it's now in a readable state [4]. We are going to submit that
>>> proposal as an additional change proposal to the main working group
>>> late today. I don't know if it has general task force approval and
>>> it's too late to do a poll for this. But we certainly want to submit a
>>> change proposal within the deadline. Others should be free to submit
>>> their own if they disagree.
>>>
>>> Cheers,
>>> Silvia.
>>>
>>>
>>> [1] http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API#.2810.29_HTML_Accessibility_Task_Force_proposal_-_.22The_San_Diego_Thought_Experiment.22
>>> [2] http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API#.286.29_Synchronize_separate_media_elements_through_attributes
>>> [3] http://lists.w3.org/Archives/Public/public-html/2011Mar/0436.html
>>> [4] http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Change_Proposal
>>>
>>>
>>>
>>
>>
>
>
>

Received on Sunday, 27 March 2011 22:47:49 UTC