- From: Janina Sajka <janina@rednote.net>
- Date: Sun, 20 Mar 2011 22:22:54 +0000
- To: HTML Accessibility Task Force <public-html-a11y@w3.org>
Minutes from Saturday's Media session are provided below in text.
They're also available as html at:
http://www.w3.org/2011/03/19-media-minutes.html
W3C
- DRAFT -
Media breakout, HTML Accessibility Task Force Face to Face
19 Mar 2011
Agenda
See also: IRC log
Attendees
Present
FtF, F2F
Regrets
Chair
Janina_Sajka
Scribe
lwatson, janina
Contents
* Topics
1. Review of Use Cases on ISSUE-152 multitrack media
* Summary of Action Items
__________________________________________________________________________________________________________________
<silvia> http://lists.w3.org/Archives/Public/public-html-a11y/2011Mar/0131.html
<silvia> bug on text alternatives for video: http://www.w3.org/Bugs/Public/show_bug.cgi?id=12228
<Sean> test
Review of Use Cases on ISSUE-152 multitrack media
<silvia> http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API
<silvia> http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_Rendering
<lwatson> scribe: lwatson
<dboudreau> test
<silvia> http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_Rendering
<JF> + 1. Review of Use Cases
<JF> + 2. JS API for in-band and manifests
<JF> + 3. HTML markup for external tracks
<JF> + 4. CSS and rendering
<JF> + 5. Error handling for multitrack and track (see
<JF> http://www.w3.org/Bugs/Public/show_bug.cgi?id=8657)
<JF> + 6. Event handling for <track(see
<JF> http://www.w3.org/Bugs/Public/show_bug.cgi?id=8659 and
<JF> http://lists.w3.org/Archives/Public/public-html-a11y/2011Mar/0018.html)
<JF> + 7. Navigation between tracks, see
<JF> http://www.w3.org/html/wg/tracker/issues/163
SP: It is a collection of images that show how multi-track video can be displayed.
... The question is what use cases do we want to support?
... There are two videos in parallel, they are independent but synchronised. Is this is something we want to consider?
SH: You may want to use this for duplicating content in sign language.
EC: If we want to support multi-track, it doesn't matter if it's co-related.
... If we want to be able to support picture and picture/synchornising multiple video streams, the presentation issue a
separate issue.
SP: There will be challenges. Should there be one set of controls or two?
EC: In the image, the side/side videos have their own controls. IMHO, we should not try to support this. This could be
done through script...
SP: Except for the synchronisation part.
SH: In this example, are they playing the same tune?
... If you want it to ound good, it needs to be synchronised properly.
SP: A solution for snychronisation would be good.
SH: Is that an accessibility issue or not?
JF: Yes, absolutely.
<silvia> EC: I think that having independent video elements on the page and synchronizing them through a mechanism as a
solution to accessibility needs (audio description & sign language video) is too difficult to authors to deal with,
e.g. having to write custom CSS and custom JS for it
<silvia> EC: I strongly believe we have to make it simple for authors to provide this a11y content
<silvia> SP: so we cannot provide a solution to a11y needs by providing multiple video elements on a page and
synchornizing them to each other
<silvia> JF: we need both, in-band and out of band support
<silvia> FO: can we solve the out-of-band with JAvaScript? how closely synchronized do things need to be?
<silvia> SP: not good enough - I have an example that shows how stongly the elements go out of sync
<silvia> EC: yes, not good enough - what about scrubbing etc
<silvia> SH: you can do it in JS through the event, unless you want to do in-sync audio
<silvia> audio-level synchronization cannot be done in JS, so we need a solution for it
<silvia> EC: we need markup so we have combined controls
<silvia> Summary of our use case discussion:
<silvia> (1) we need to support in-band multitrack media, e.g. sign language, audio description, language dubs provided
in the same resource in-band
<silvia> (2)
<silvia> we need to support out-of-band mutltirack media which are tightly coupled, in that they require a single
control, where the main resource is a master and the markup needs to be simple without requiring extra CSS to display
them as though they were in-band
<silvia> (3) we may want to support out-of-band multitrack media which are loosely coupled, in that they are separate
elements on the page each with their own controls, but interacting with one means the other(s) follow
<silvia> guidance for developing the synchronization:
<silvia> * video should be synchronized for sign language to 0.5sec accuracy
<silvia> * audio should be synchronized for audio descriptions to 100ms resolution
<silvia> * we want to achieve a consistent API between in-band and external audio/video tracks
<silvia> http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API
<silvia> * we want to be able to control the relative volume of a additional audio track by user and author
<silvia> * we want to be able to control positioning of video tracks as picture-in-picture or side-by-side viewports in
script by author
<silvia> * we don't want to inflict more complexity on the <source> selection, which is based on choice of encoding
formats, but we can replicate that somewhere else
<silvia> * the same source selection algorithm needs to be applied to all choices of media encoding formats, so it is
predictable to the author
<silvia> * we want to satisfy the 80% case - e.g. synchronizing an extended audio description with an un-extended
original video is not what we want to do here (you can do that by also using a TimedTrack and JS)
<silvia> ACTION 2. JS API for in-band and manifests
<frankolivier> For our proposal: We're happy to have a generic JS API for enumerating/enabling inband and non-inband
tracks
JF: It seems that changing CSS is a bigger challenge.
EC: I think it woul be a mistake to do it in the markup. It's requiring more from the author, and it has the same issue
that people complain about with having to keep markup in synch with the format of the external file.
... We'd be better off working with the CSS I think.
SP: I agree that whe the tracks are inside the resource you already have the whole description.
JF: Should we invite someone from CSS WG into the discussion?
SP: Let's propose our suggestion and see what happens.
EC: It's a way to style in band and out band tracks using pseudo selectors.
SP: Can we agree we don't need extra markup for in band tracks?
<silvia> * default track selection should come from the container in the in-band case
<silvia> FO: when we talk about track, do we mean audio, video *and* text?
<silvia> SP: basically anything time-synchronized data
<silvia> FO: any additional content that supplements the main audio/video track in time-sync
<silvia> TOPIC 2: JS API for in-band and manifests
<janina> scribe: janina
<silvia>
http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API#.281.29_No_markup_in_HTML_-_leave_to_a_manifest_file
<silvia>
http://www.w3.org/WAI/PF/HTML/wiki/Media_Multitrack_Media_API#.289.29_Audio_Track_Selection_for_Media_Element_.2F_In-ba
nd_only
ec: we want to end up with the same api for in and out of band media--eventually
<silvia> interface HTMLMediaElement : HTMLElement {
<silvia> [...]
<silvia> // audio tracks
<silvia> readonly attribute unsigned long audioTrackCount;
<silvia> readonly attribute DOMString audioTrackLanguage[];
<silvia> attribute unsigned long currentAudioTrack;
<silvia> };
<silvia> ==
<silvia> interface MediaTrack {
<silvia> readonly attribute DOMString kind;
<silvia> readonly attribute DOMString label;
<silvia> readonly attribute DOMString language;
<silvia> const unsigned short OFF = 0;
<silvia> const unsigned short HIDDEN = 1;
<silvia> const unsigned short SHOWING = 2;
<silvia> attribute unsigned short mode;
<silvia> }
<silvia> interface HTMLMediaElement : HTMLElement {
<silvia> [...]
<silvia> readonly attribute TextTrack[] textTracks;
<silvia> readonly attribute MediaTrack[] mediaTracks;
<silvia> };
fo: what does the author do with kind?
ec: should be a defined list to select from
sp: everything we make available we need also api for custom controls
discussion of referring to external tracks inband
mpeg4 supports this via manifest
sp: api has onload and onerr ... keep? drop?
fo: keep symetry between in and out of band -- to support authoring flexibility
ec: authors shouldn't have to care
<silvia> basing discussion on second interface pasted above since it is more generic
<silvia> need to add back in all the IDL attributes and functions of the TextTrack API, because authors should not need
to care where the data comes from
<silvia> in particular events should be identical so registration would not depend on the type of track
<silvia> add back in...
<silvia> attribute http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#function
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#handler-texttrack-onload;
<silvia> attribute http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#function
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#handler-texttrack-onerror;
<silvia> const unsigned short http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-none
= 0;
<silvia> const unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-loading = 1;
<silvia> const unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-loaded = 2;
<silvia> const unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-error = 3;
<silvia> readonly attribute unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-readystate;
<silvia> readyState may not mean much and my be a reflection of the state from the main resource
<silvia> we have complications on cues:
<silvia> readonly attribute http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#texttrackcuelist
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-cues;
<silvia> readonly attribute http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#texttrackcuelist
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-activecues;
<silvia> attribute http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#function
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#handler-texttrack-oncuechange;
<silvia> is there a need for audio and video for these?
<silvia> ec: we may need to subclass similar to how HTMLMediaElement works
<silvia> http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#text-track-api
<silvia> FO: deriving from base class makes sense, but what if we have data that is different to audio/video/text?
<silvia> ⊠won't we restrict ourselves too much?
<silvia> EC: no, because we'll have mediatrack and that is more generic
<silvia> interface HTMLMediaElement : HTMLElement {
<silvia> [...]
<silvia> readonly attribute TextTrack[] textTracks;
<silvia> readonly attribute MediaTrack[] mediaTracks;
<silvia> };
Thanks, Michael. Is your phone OK?
<silvia> SP: does it make more sense to have audioTracks and videoTracks?
<silvia> EC: we don't want a proliferation of attributes ⊠so I'm torn
<silvia> EC: I think I prefer mediaTracks, because the script can figure out what type it is
<silvia> SP: do we need a mime type or something?
<silvia> EC: we introduced mediaType into proposal 8
<silvia> FO: I'm concerned we need different interfaces for audio and video
<silvia> EC: yes, I think we need more attributes
<silvia> EC: we have two levels - media is the parent, audio, video and text derive from that
<silvia> SP: hmm, do we then even need textTracks?
<silvia> EC: no, we only need tracks then as an attribute in HTMLMediaElement
<silvia> SP: are we replicating attributes from the parent?
<silvia> EC: no we should not, only some select ones
<silvia> FO: eveything that has to do with loading and buffering need to be replicated, but anything to do with time
should not
<silvia> .. or playbackrate
<silvia> SH: what about src and currentSrc - would you be able to change them?
<silvia> SP: for in-band it doesn't make sense
<silvia> EC: for out-of-band we shouldn't make this available either
<silvia> FO: we need src to provide the original url
<silvia> EC: it should be read-only
<silvia> .. can't be read-only because you need to set it once
<silvia> FO: why would you ever want to change the src of a media track?
<silvia> EC: for example if you have a playlist
<silvia> SP: you could completely remove the video element and re-build the whole thing
<silvia> EC: hmm, you already have to handle all the work when it's set the first time, so changing it on the fly
should be similar
<silvia> .. the ready state is available to both
<silvia> FO: we should make the track elements behave a lot like the media elements
<silvia> agreement in general that we should have only one tracks attribute to add to the HTMLMediaElement
<silvia> SH: we need a separate way to set volume - need an audioTrack element
<silvia> SP: what to do with video that also has an audio track?
<silvia> EC: I don't think we should offer flow control at the track level
<silvia> SH: would audio and video behave independently?
<silvia> EC: they should behave like there are two separate tracks
<silvia> interface MediaTrack {
<silvia> readonly attribute DOMString
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-kind;
<silvia> readonly attribute DOMString
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-label;
<silvia> readonly attribute DOMString
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-language;
<silvia> const unsigned short http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-none
= 0;
<silvia> const unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-loading = 1;
<silvia> const unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-loaded = 2;
<silvia> const unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-error = 3;
<silvia> readonly attribute unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-readystate;
<silvia> attribute http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#function
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#handler-texttrack-onload;
<silvia> attribute http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#function
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#handler-texttrack-onerror;
<silvia> const unsigned short http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-off
= 0;
<silvia> const unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-hidden = 1;
<silvia> const unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-showing = 2;
<silvia> attribute unsigned short
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-mode;
<silvia> [these are the attributes from the TextTrack that make general sense]
<silvia> it would be nice to unify the states in the "readyState" here, because they apply uniformly for audio, video
and text tracks
<silvia> we need to add the @default attribute in the IDL:
<silvia> readonly attribute DOMString default;
<silvia> SH: I would prefer to be able to turn multiple on by default
<Sean> have a showing attribute, rahter than a default
<silvia> mode="showing" is more flexible
<silvia> SH: what we don't want is two attributes that influence the same IDL
<silvia> SH: there is no way from markup to say "download this but don't show it"
<silvia> FO: you need two concepts
<silvia> ⊠for mobile devices you need to have some sort of preload attribute that says "don't download this data at
all"
<silvia> ⊠similarly you need to have the concept of requiring download of all the necessary track data before you
allow playbac
<silvia> k
<silvia> EC: I think the description of tracks right now requires captions to be downloaded before playback - the
element can't reach HAVE_METADATA before they have loaded
<frankolivier> .
<silvia> SP: do we need information on whether we want alternative data to be loaded before going to HAVE_METADATA?
<silvia> FO: if we use mode, then we can do it: OFF means not to wait and SHOWING would require to wait
<silvia> ⊠HIDDEN would allow starting playback while in the background loading the resource
<silvia> s/HIDDEN or//
<silvia> SP: so do we want to remove the @default attribute and instead have a @mode attribute with values in {off,
hidden, showing} ?
<silvia> EC: what if I just want one as the default?
<silvia> SP: then just set the attribute on that to @mode="showing"
<silvia> EC: what if the UA need something else as the default from settings - would it overwrite what the author set?
<silvia> FO: are you allowed as the UA to overwrite the value of @mode ?
<silvia> FO: same thing with the windows-wide setting for showing captions/subtitles
<silvia> SP: if the author e.g. set the english track to @mode="showing", but my UA settings say to grab the french
ones, then the UA should set the english track to @mode="hidden" and the french one to @mode="showing"
<silvia> SH: we then also need an event onchangemode
<silvia> EC: what if we have a situation where we need multiple tracks showing, or one always on, such as in a language
lab?
<silvia> SP: the 80% use case I think is that we have only one track of one type active at one time
<silvia> ⊠if we need more than one track active, we can always use script
<silvia> FO: we can do multiple tracks through multiple @mode attributes already
<silvia> SH: what if the UA can just turn on additional ones
<silvia> .. then have to turn them off
<silvia> SP: not really useful
<silvia> JF: what if we have english and french tracks, but my UA settings are spanish
<silvia> FO: if there is no track in my language, then nothing should be shown
<silvia> EC: or the first track - it needs to be predictable
<silvia> FO: only overwrite an author's settings if nothing is specified
<silvia> EC: leave the default choice to the browser
<silvia> FO: what if the controls attribute it set and the menu is showing - what then?
<silvia> EC: right-click menu
<silvia> FO: if the @controls is not set, is the user allowed to fiddle with the state of the media resource
<silvia> EC: I think yes, and all the basic controls should be available in the context menu
<silvia> FO: so my custom controls need to deal with it?
<silvia> EC: if I as a conscientious author would mark them up so the UA chooses, would I mark up all tracks as
@mode="off"?
<silvia> SP: no, I wouldn't provide @mode at all
<silvia> EC: for audio and video it makes a big difference if hidden also means preloading
<silvia> FO: so, there is no @default attribute, but the default mode on all tracks is OFF
<silvia> SP: do we need src, currentSrc and load() next?
<silvia> http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#attr-media-src
<silvia> EC: don't think we need load() for changing the url, because the spec now says to run the source selection
algo upon changing
<silvia> ⊠load() is just used to reset state of the video element
<silvia> ⊠nothing will happen until the script has gone back to idle
<silvia> ⊠resetting only needs to be available on main video
<silvia> ⊠but you can change the src
<silvia> SH: video.load() should filter down to any tracks
<silvia> SP: so, if I change the @src on a track, then the UA should load it and jump to the time offset of
video.currentTime
<silvia> ⊠so that is additional work to what is in the general media element load algorithm
<silvia> ⊠it is similar to setting startOffset on main resource, but that attribute should only exist on the main
resource
<silvia> SH: do we need canPlayType() on tracks?
<silvia> EC: semantically it is only required by the media element
<silvia> SP: it's only required to find out what codecs your browser supports
<silvia> ⊠so the same applies to tracks as well as <video>
<silvia> EC: is it going to be legal to use canPlayType on text tracks?
<silvia> EC: it would make sense to ask canPlayType on caption formats
<silvia> ⊠but you should not be able to set the video.src to it
<silvia> SP: we already have that problem that we don't have a function that allows us finding out what text formats
are supported
<silvia> SH: maybe we should introduce a canPlayTrackType() function
<silvia> FO: what about in-band tracks and their formats?
<silvia> EC: some do, some don't use
<silvia> ⊠the question about formats for in-band is not particularly useful
<silvia> ⊠so mime types are sufficient
<silvia> ⊠I don't think there's a mime types for 3GPP captions for example
<silvia> ⊠and if there was, they would be in the codecs parameter
<silvia> SP: are we all in agreement that we need canPlayTrackType()
<silvia> ⊠not sure I can come up with something more elegant...
<silvia> SP: do we need networkState?
<silvia> ⊠not on in-band tracks
<silvia> ⊠but external ones...?
<silvia> SP: am almost not sure if we need networkState for video from a script POV
<silvia> EC: we argued a lot whether it makes sense to have both states, readyState and networkState .. it really is
one state machine only
<silvia> .. I don't really want to add readyState to tracks
<silvia> SP: let's leave it out for now, until somebody can make a good case for it
<silvia> SP: what about error
<silvia> SH: we need some kind of error states
<silvia> EC: we need exactly the same error staes
<silvia> ⊠need to add it to MediaTrackElement
<silvia> readonly attribute http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#mediaerror
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-media-error;
<silvia> interface HTMLMediaTrack {
<silvia> readonly attribute DOMString kind;
<silvia> readonly attribute DOMString label;
<silvia> readonly attribute DOMString language;
<silvia> const unsigned short NONE = 0;
<silvia> const unsigned short LOADING = 1;
<silvia> const unsigned short LOADED = 2;
<silvia> const unsigned short ERROR = 3;
<silvia> readonly attribute unsigned short readyState;
<silvia> attribute Function onload;
<silvia> attribute Function onerror;
<silvia> const unsigned short OFF = 0;
<silvia> const unsigned short HIDDEN = 1;
<silvia> const unsigned short SHOWING = 2;
<silvia> attribute unsigned short mode;
<silvia> interface MediaTrack {
<silvia> readonly attribute DOMString kind;
<silvia> readonly attribute DOMString label;
<silvia> readonly attribute DOMString language;
<silvia> const unsigned short NONE = 0;
<silvia> const unsigned short LOADING = 1;
<silvia> const unsigned short LOADED = 2;
<silvia> const unsigned short ERROR = 3;
<silvia> readonly attribute unsigned short readyState;
<silvia> attribute Function onload;
<silvia> attribute Function onerror;
<silvia> const unsigned short OFF = 0;
<silvia> const unsigned short INACTICE = 1;
<silvia> const unsigned short ACTIVE = 2;
<silvia> attribute unsigned short mode;
<silvia> attribute DOMString src;
<silvia> readonly attribute DOMString currentSrc;
<silvia> readonly attribute MediaError error;
<silvia> }
<silvia> .. UH⊠just use the bottom half of that
<silvia> SH: which events then do we have?
<Sean> DOM EventListener interface
<Sean> this.addEventListener = function (type, handler, bubble) { // Register an event handler of a specific event type
on the EventTarget. } this.removeEventListener = function (type, handler) { //Removes an event listener from the
EventTarget. - } this.dispatchEvent = function (event) { //Dispatch an event to this EventTarget.
<silvia> EC: do we want an event interface for onload()?
<silvia> ⊠what do you do with an inband track? it will never fire onload
<silvia> SH: we need two kinds of errors - a partial error and a full error
<silvia> ⊠you need to be able to indicate that a set of cues has not been properly parsed for example
<silvia> SP: but we don't have a visual presentation of such errors in HTML either, e.g. when a piece of HTML is not
properly loaded
<silvia> SH: if an image cannot load, something is shown
<silvia> SP: yeah, but not for html text
<silvia> SH: if something goes wrong, you may want to load a different resource...
<silvia> EC: the MediaError state has these states: includes ERROR
<silvia> SP: so partial and full errors would turn up in the error state
<silvia> .. as MEDIA_ERR_SRC_NOT_SUPPORTED or MEDIA_ERR_DECODE
<silvia> EC: agreed
<silvia> EC: back to events
<silvia> ⊠if we are going to support streamed captions, we cannot require playback to be paused because captions
haven't loaded
<silvia> SP: what about just loading the first resource
<silvia> EC: what if it comes really late
<silvia> SP: what about just HAVE_METADATA related state?
<silvia> EC: yes, that seems to make sense
<silvia> SP: so let's replace the readyState with the proper ones from the media element:
<silvia> const unsigned short HAVE_NOTHING = 0;
<silvia> const unsigned short HAVE_METADATA = 1;
<silvia> const unsigned short HAVE_CURRENT_DATA = 2;
<silvia> const unsigned short HAVE_FUTURE_DATA = 3;
<silvia> const unsigned short HAVE_ENOUGH_DATA = 4;
<silvia> readonly attribute unsigned short readyState;
<silvia> SH: make sure we change the parsing algorithm for TextTrack to throw an error on faulty parsing
<silvia> MK: if a track element has an error, would that affect the parent?
<silvia> FO: probably not because the tracks have their own error states
<silvia> SP: would it stop playing?
<silvia> FO: no, the main resource should continue playing
<silvia> SH: we have an onerror event to allow the author to deal with error situations
<silvia> EC: but we cannot have an onload event
<silvia> SH: we need an information that the track is proceeding
<silvia> SP: I think that unless we get an onerror on a track, we can assume that all tracks continue loading
<silvia> .. and can be played
<silvia> EC: yes, for progress and timeupdate events
<silvia> SH: if we get an error from a track, then we know something stopped working
<silvia> ⊠we can register a single event handler for all tracks
<silvia> EC: the target shows where it was raised from
<silvia1> SP: what do we do when external audio and video tracks stop buffering or fail half-way through
<silvia1> ⊠e.g. if audio descriptions suddenly fail - would we want to have the main audio continue playing?
<silvia1> JS: I would want to react differently depending on the situation
<silvia1> ⊠with friends, it should probably continue
<silvia1> ⊠with a silent movie, probably not
<silvia1> ⊠if it continued playing, I might assume that something is wrong with my audio device
<silvia1> EC: might be best to leave that problem to the UA to solve
<mkobayas> ping
<silvia1> interface textTrack : mediaTrack {
<silvia1> readonly attribute TextTrackCuesList cues; readonly attribute
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#texttrackcuelist
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#dom-texttrack-activecues;
<silvia1> attribute http://www.whatwg.org/specs/web-apps/current-work/multipage/webappapis.html#function
http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#handler-texttrack-oncuechange;
<silvia1> interface textTrack : mediaTrack {
<silvia1> readonly attribute TextTrackCuesList cues;
<silvia1> readonly attribute TextTrackCueList activeCues;
<silvia1> attribute Function oncuechange;
<silvia1> };
<silvia1> (just look at the last half of this)
<silvia1> FO: why do we need to fire events for every cue?
<silvia1> ⊠oncuechange is not helpful - it's better to know whether something has been added / removed
<silvia1> ⊠we could just add the cues that are going away and are starting to the cuechange handler
<silvia1> FO: would inactive tracks still fire events?
<silvia1> EC: if you want to render them yourself, then you don't want them showing, so what do you do?
<silvia1> SH: what is the getCueAsText returning? should it return just innterHTML?
<silvia1> s/innterHTML/innerText/?
<silvia1> .. since that function already exists, maybe we just need getAsHTML and can then put getInnterText on it
<silvia1> EC: but you may not be able to get on the text at all
<silvia1> ⊠it can display
<silvia1> SH: the rules say that if the captions aren't coming from the same domain, you cannot display them at all
<silvia1> EC: if it's only text, it shouldn't be a problem
<silvia1> SH: because it throws all the events etc, so only turning off getCueAsHTML and getCueAsText is not sufficient
<silvia1> SP: CORS is supposed to solve this
<silvia1> EC: there is some information leakage with video, but you can't get any video data out of it
<silvia1> ⊠it works for video, so it should probably work on captions
<silvia1> FO: we need to have this discussion in the HTML WG
<silvia1> ⊠it's a security discussion
<silvia1> http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#texttrackcue
<silvia1> http://www.whatwg.org/specs/web-apps/current-work/multipage/video.html#texttrack
<silvia1> SH: you want onenter and onexit on the track, not on the cue, because you only want one handler to deal with
the changes
<silvia1> SH: why don't we have cues show up in the DOM?
<silvia1> SP: I think it had something to do with security
<silvia1> ⊠but seeing as it is same-site content only, I don't know how that makes sense any more
<silvia1> FO: it would also be possible to load them all into the DOM and then you can just make them visible when the
time is right
<silvia1> EC: wouldn't work for streamed catpions
<silvia1> FO: we like the idea of having captions in the DOM
<silvia1> [group has a discussion on #whatwg about why putting cues directly into the DOM is a bad idea]
<silvia1> see http://krijnhoetmer.nl/irc-logs/whatwg/20110320#l-77
Summary of Action Items
[End of minutes]
__________________________________________________________________________________________________________________
--
Janina Sajka, Phone: +1.443.300.2200
sip:janina@asterisk.rednote.net
Chair, Open Accessibility janina@a11y.org
Linux Foundation http://a11y.org
Chair, Protocols & Formats
Web Accessibility Initiative http://www.w3.org/wai/pf
World Wide Web Consortium (W3C)
Received on Sunday, 20 March 2011 22:23:23 UTC