- From: <bugzilla@jessica.w3.org>
- Date: Tue, 25 Sep 2012 21:54:23 +0000
- To: public-html-a11y@w3.org
https://www.w3.org/Bugs/Public/show_bug.cgi?id=19020
Summary: <audio> and <video> do not have sufficient support for
synchronized accessibility content
Product: HTML WG
Version: unspecified
Platform: All
OS/Version: All
Status: NEW
Keywords: a11y, a11ytf, media, PFWG, TrackerIssue
Severity: normal
Priority: P3
Component: HTML5 spec
AssignedTo: dave.null@w3.org
ReportedBy: contributor@whatwg.org
QAContact: public-html-bugzilla@w3.org
CC: mjs@apple.com, mike@w3.org,
laura.lee.carlson@gmail.com,
public-html-wg-issue-tracking@w3.org,
public-html@w3.org, jimjjewett@gmail.com,
shadow2531@gmail.com,
xn--mlform-iua@xn--mlform-iua.no, cooper@w3.org,
w3c@kliehm.com, public-html-a11y@w3.org,
silviapfeiffer1@gmail.com, caldwell@trace.wisc.edu,
singer@apple.com, a.kuckartz@ping.de
This was was cloned from bug 5758 as part of operation LATER convergence.
Originally filed: 2008-06-15 01:36:00 +0000
Original reporter: Jim Jewett <jimjjewett@gmail.com>
================================================================================
#0 Jim Jewett 2008-06-15 01:36:22 +0000
--------------------------------------------------------------------------------
The new <audio> and <video> elements do not permit fallback content.
This is a regression from the more generic <object> (though not from <embed>).
The reasoning is that authors SHOULD use accessibility aids built into the
underlying media. While this is indeed desirable, it is not always possibe --
and relies on the browser being able to interpret the media stream, at least to
some extent.
An additional fallback step is required for authors who cannot modify the
original stream, and for users of browsers which do not understand the format
well enough to extract accessibility information.
The fallback already available for <object> would be sufficient, as would a
more specialized element such as the sometimes proposed <alt>.
================================================================================
#1 Ian 'Hixie' Hickson 2008-06-15 02:12:35 +0000
--------------------------------------------------------------------------------
<video> and <audio> do support fallback content for user agents with no video
and audio support. Just put the fallback contents in the elements.
We will have a format that all UAs support, and that will be the fallback to
use when this is deployed. So there is no concern of lack of codec support.
For users who really don't want audio/video but want an alternative form,
fallback support wouldn't work, since the UA would quite happily just show the
video. Instead, authors should just provide links to transcripts (or whatever).
This isn't an accessibility concern (or at least isn't exclusively a concern
for disabled users, I should say), because all users sometimes want
transcripts.
================================================================================
#2 Jim Jewett 2008-06-15 14:21:29 +0000
--------------------------------------------------------------------------------
(1) The standard does not require use of the "one good codec"; other codecs
are explicitly permitted.
(2) The "one good codec" has not yet been chosen. That is largely because
there are no such codecs yet -- and that is even without an explicit
accessibility requirement on the codec selection.
(3) There will be user agents that cannot do audio or video because of
hardware limits; expecting them to download the entire multimedia stream just
to extract the accessibility data is perhaps too optimistic.
================================================================================
#3 Jim Jewett 2008-10-17 02:20:59 +0000
--------------------------------------------------------------------------------
Also note that it is not always possible to edit the original media. If the
media comes from a third party, the web page author may not have the either the
legal authority to modify it (copyright concerns) or the technical ability
(offsite hosting; proprietary formats). The web page author must still be able
to provide fallback content as appropriate to the web page.
================================================================================
#4 Ian 'Hixie' Hickson 2008-11-22 12:15:33 +0000
--------------------------------------------------------------------------------
"Fallbacks" like transcripts are suitable for everyone, so it doesn't make
sense to hide them when the video is available. Transcripts that are
alternative media streams should just be listed as <source>s. What other kinds
of fallback did you have in mind?
================================================================================
#5 Martin Kliehm 2009-10-29 15:24:18 +0000
--------------------------------------------------------------------------------
I cannot tell what the original aim of the bug reporter were, but regarding
accessible fallback this is what WCAG guideline 1.2 is about.[1]
1) If a fallback element is nested within the audio or video element, in some
cases the fallback element still needs to be focusable by keyboard, even when
the browser supports audio/video. For example Bruce Lawson [2] nests a download
link within a video element, so this should be keyboard accessible.
2) If the audio or video content itself is inaccessible due to the nature of
the person's disability, accessible alternative content should be available.
Accessible alternative content could be a transcript, a textual description, an
audio description, captions, or a synchronized sign language video. This
content could be linked or otherwise machine-readable bolted onto the element
by attributes. Machine-readability is important for assistive technology like
screenreaders, but also for search engines. @aria-describedby comes to my mind
for text, but @alt or @longdesc could also be appropriate. Synchronized
multimedia formats need other attributes to be associated with the original
audio/video.
For example a link to a transcript would be sufficient, but users should be
able to select if captions are displayed, choose from a range of several
captions in multiple languages, and if a sign language interpretation is
available this could be displayed as picture-in picture. Bruce Lawson writes
that he expected the @controls to specify this, alas this is not yet the
case.[2]
It should be noted that accessing alternative content via controls is important
for all users, so it would be counter-productive to make them only available if
assistive technology devices are present. Deaf users just need a way to enable
captions, and so do users who don't have any audio equipment.
Regarding captions there are several competing formats: DFXP [3] as a W3C
standard, YouTube officially supports SubViewer and SubRip, inofficially also
SAMI and SMIL, but there's also DAISY, QTtext, DVB, TimedText, EBU, SCC, Kate,
CMML, SSA, MicroDVD, USF, and VOBSub, among others. I would expect browser
vendors to agree on a set of supported formats.
>From an author's perspective there need to be easy methods to add such content;
editing a link or adding an attribute that refers to another file is
reasonable, but embedding captions within a video file could be beyond their
abilities.
[1] http://www.w3.org/TR/WCAG/#media-equiv
[2]
http://www.brucelawson.co.uk/2009/accessibility-of-html5-video-and-audio-elements/
[3] http://www.w3.org/TR/ttaf1-dfxp/
================================================================================
#6 Martin Kliehm 2009-10-29 21:02:32 +0000
--------------------------------------------------------------------------------
Example:
- BBC video with timed text, synchronized to chapters:
http://open.bbc.co.uk/rad/demos/html5/rdtv/episode2/index.html
But it doesn't end here. What we will see often is a combination of video and
canvas. Although a video element might be present what we actually see could be
video in a canvas. Just bear that in mind when solving bug #7011.
1. Content Injection:
http://people.mozilla.com/~prouget/demos/DynamicContentInjection/play.xhtml
2. Filters:
https://developer.mozilla.org/En/Manipulating_video_using_canvas
http://blip.tv/file/1881590
3. Face recognition and social media layers:
http://blip.tv/file/2265515
4. Dailymotion (huge French video site) demo in combination with JavaScript,
SVG, CSS3, and canvas control elements (volume slider):
http://www.dailymotion.com/openvideodemo
================================================================================
#7 Michael Cooper 2009-11-03 21:21:15 +0000
--------------------------------------------------------------------------------
PFWG is interested in this issue. A group has formed to explore the
requirements and solutions. A preliminary meeting was held 1 November 2009
<http://www.w3.org/2009/11/01-media-minutes>. The work will continue in the
HTML Accessibility Task Force. Note ISSUE-9 (currently "raised") is also
relevant.
================================================================================
#8 Laura Carlson 2009-11-04 00:13:48 +0000
--------------------------------------------------------------------------------
Multimedia Accessibility <Audio> <Video> Wiki Page
http://esw.w3.org/topic/HTML/MultimediaAccessibilty
================================================================================
#9 Ian 'Hixie' Hickson 2010-01-04 08:33:00 +0000
--------------------------------------------------------------------------------
(In reply to comment #5)
> 1) If a fallback element is nested within the audio or video element, in some
> cases the fallback element still needs to be focusable by keyboard, even when
> the browser supports audio/video. For example Bruce Lawson [2] nests a download
> link within a video element, so this should be keyboard accessible.
That's a user agent implementation issue. Since the children of <video> are
only exposed to users of UAs that don't support <video> at all, the HTML5 spec
can't do much about it anyway if it could, the UAs would just implement
<video>.
> 2) If the audio or video content itself is inaccessible due to the nature of
> the person's disability, accessible alternative content should be available.
This is possible (indeed, easy): just include such content, or links to such
content, in the page (not inside the <video> element).
> Accessible alternative content could be a transcript, a textual description
Both of these are already supported, just by regular HTML alongside the
<video>. No special feature is needed to enable this.
> an audio description, captions, or a synchronized sign language video.
These are features of the video formats themselves. We haven't yet resolved the
format issue. If we can't resolve it, and indeed even if we can, in due course,
once browsers implement what is already specced to an adequate level, we will
be introducing explicit syntax for time-synchronised captions, sign language
insets, etc. This isn't fallback, however, so is out of scope for this bug.
> This
> content could be linked or otherwise machine-readable bolted onto the element
> by attributes. Machine-readability is important for assistive technology like
> screenreaders, but also for search engines. @aria-describedby comes to my mind
> for text, but @alt or @longdesc could also be appropriate. Synchronized
> multimedia formats need other attributes to be associated with the original
> audio/video.
It's unclear which of the many features discussed above this is referring to.
Each of the features above have different needs, so it would be inappropriate
to address them all the same way.
> For example a link to a transcript would be sufficient, but users should be
> able to select if captions are displayed, choose from a range of several
> captions in multiple languages, and if a sign language interpretation is
> available this could be displayed as picture-in picture. Bruce Lawson writes
> that he expected the @controls to specify this, alas this is not yet the
> case.[2]
It's unclear exactly what is missing here. What is stopping user agents from
supporting captions today, for instance?
This is out of scope of this bug, though, which is about fallback. I would
encourage you to file very specific bugs on each specific issue, rather than a
single bug for the entire area.
> It should be noted that accessing alternative content via controls is important
> for all users, so it would be counter-productive to make them only available if
> assistive technology devices are present. Deaf users just need a way to enable
> captions, and so do users who don't have any audio equipment.
Indeed. This is possible today, and needs no author involvement.
> Regarding captions there are several competing formats: DFXP [3] as a W3C
> standard, YouTube officially supports SubViewer and SubRip, inofficially also
> SAMI and SMIL, but there's also DAISY, QTtext, DVB, TimedText, EBU, SCC, Kate,
> CMML, SSA, MicroDVD, USF, and VOBSub, among others. I would expect browser
> vendors to agree on a set of supported formats.
Format decisions are an outstanding issue currently, and there is work ongoing
to resolve it.
> From an author's perspective there need to be easy methods to add such content;
> editing a link or adding an attribute that refers to another file is
> reasonable, but embedding captions within a video file could be beyond their
> abilities.
On the medium term, we will be adding features for external captions and the
like. However, it would be harm accessibility of <video> on the Web on the
short and medium term if we were to add it today, before browsers have
correctly implemented the existing requirements. When browser vendors work on
this feature, we want them to be focused on it we don't want it to be just
one more item on a long list of features that they have to all implement at
once to get <video> support.
In conclusion, I think the spec as is has suitable support for _fallback_ for
<video> and <audio>, though I agree that it is far from a comprehensive
solution. I do not think, however, that it is in anyone's interests for us to
add more features to make this simpler at this time, since they would end up
misimplemented and would unintentionally harm accessibility. This doesn't just
apply to these accessibility enhancements, by the way; we've delayed a whole
bunch of features that make certain effects simpler while we wait for
implementations to catch up.
Also, it's worth noting that as it stands today, authors are very unlikely to
make immediate extensive use of <video> anyway. We haven't even found a common
codec yet. Therefore in practice the lack of easy-to-use accessibility hooks
does not actually have much of an effect at this time.
Here is what I propose as a way forward for this general topic:
- UAs continue to implement <video> such that the implementations reach a high
enough caliber that there are no longer major bugs that would distract from
additional feature work.
- in the meantime, we experiment with finding the right API, markup, and
formats for captions and other time-linked annotation data.
- for the few early-adopter authors making immediate use of <video>, we
encourage either the use of scripts that layer subtitles (etc) over video, or
the use of subtitles embedded within the video files as separate text tracks.
- once UAs have stable high-quality implementations of the feature set in the
spec today, we add external subtitle (etc) support to the spec.
EDITOR'S RESPONSE: This is an Editor's Response to your comment. If you are
satisfied with this response, please change the state of this bug to CLOSED. If
you have additional information and would like the editor to reconsider, please
reopen this bug. If you would like to escalate the issue to the full HTML
Working Group, please add the TrackerRequest keyword to this bug, and suggest
title and text for the tracker issue; or you may create a tracker issue
yourself, if you are able to do so. For more details, see this document:
http://dev.w3.org/html5/decision-policy/decision-policy.html
Status: Partially Accepted
Change Description: none at this time
Rationale: see above
================================================================================
#10 Laura Carlson 2010-01-12 13:28:10 +0000
--------------------------------------------------------------------------------
Could "Resolution: LATER" be defined for this bug?
Is there a ballpark date? Does it mean before HTML5 last call? Or does it mean
HTML6?
Thanks.
================================================================================
#11 Ian 'Hixie' Hickson 2010-01-12 21:49:21 +0000
--------------------------------------------------------------------------------
It depends on implementations, as described in detail in comment 9. The best
way to help implementations reach a reliable interoperable state faster is to
write test cases and file bugs on implementations.
================================================================================
#12 Michael Cooper 2010-02-04 16:36:15 +0000
--------------------------------------------------------------------------------
HTML Accessibility Task force making this a formal tracked bug for the TF.
================================================================================
#13 Michael Cooper 2010-02-04 16:42:37 +0000
--------------------------------------------------------------------------------
Removing myself from cc list, default on bug edits is to add, but I'm only
acting on behalf of HTML A11Y TF which is already cc'd.
================================================================================
#14 Silvia Pfeiffer 2010-02-07 01:36:15 +0000
--------------------------------------------------------------------------------
The discussions on this bug have touched multiple topics related to media and
accessibility.
I'd like to propose this bug to focus on the support of synchronised
accessibility data (captions, subtitles, audio description, sign language) -
both embedded in stream and external.
In particular, this bug is not about "fallbacks" (and possibly the word
"fallback" should be replaced with "support" in the title).
As for all the other topics that this bug has touched upon, here is what I took
out:
* choice of a baseline code - not something we can make any progress on through
a bug
* video fallback in UAs where video is not supported - already possible
* non-synchronised text alternatives - already possible through putting text or
a link next to the video (and possibly using aria-describedBy to link it to the
video)
* access to captions/subtitles available in media files that a UA does not
support (e.g. MPEG with subtitles in Firefox) - is simply not technically
possible
* @alt attribute - has been discussed in WHATWG, see thread at
http://lists.whatwg.org/htdig.cgi/whatwg-whatwg.org/2009-August/021865.html;
basically, the following attributes already provide what is required: @title,
@aria-label, @aria-describedby
Hope this answers some questions and lets us move to the core accessibility
issues for media elements that are still open and that Ian proposed a process
for how to address.
================================================================================
#15 Maciej Stachowiak 2010-03-14 13:14:17 +0000
--------------------------------------------------------------------------------
This bug predates the HTML Working Group Decision Policy.
If you are satisfied with the resolution of this bug, please change the state
of this bug to CLOSED. If
you have additional information and would like the editor to reconsider, please
reopen this bug. If you would like to escalate the issue to the full HTML
Working Group, please add the TrackerRequest keyword to this bug, and suggest
title and text for the tracker issue; or you may create a tracker issue
yourself, if you are able to do so. For more details, see this document:
http://dev.w3.org/html5/decision-policy/decision-policy.html
This bug is now being moved to VERIFIED. Please respond within two weeks. If
this bug is not closed, reopened or escalated within two weeks, it may be
marked as NoReply and will no longer be considered a pending comment.
================================================================================
#16 Michael[tm] Smith 2010-03-14 13:33:02 +0000
--------------------------------------------------------------------------------
http://www.w3.org/html/wg/tracker/issues/9
================================================================================
#17 Jim Jewett 2010-03-15 13:54:46 +0000
--------------------------------------------------------------------------------
It is formally on me to accept resolution and close, or not.
Given comment 12, I don't feel comfortable doing that.
I won't claim to be fully satisfied, but I'm not sure how to better express my
lingering concerns -- so I defer to the HTML Accessibility Task. When they're
satisfied, I'll agree to closure.
================================================================================
#18 Maciej Stachowiak 2010-03-15 14:42:53 +0000
--------------------------------------------------------------------------------
This bug is already marked TrackerIssue so there's no need to close it right
now.
================================================================================
#19 Michael[tm] Smith 2010-03-18 15:45:11 +0000
--------------------------------------------------------------------------------
Reopened per discussion on a11y TF telcon 2010-03-18
================================================================================
#20 Maciej Stachowiak 2010-03-18 16:43:22 +0000
--------------------------------------------------------------------------------
Since this bug has already been escalated to the tracker, it's inappropriate to
also reopen it.
================================================================================
#21 Maciej Stachowiak 2010-03-18 16:43:41 +0000
--------------------------------------------------------------------------------
(Moving back to VERIFIED)
================================================================================
#22 Laura Carlson 2010-03-22 16:20:52 +0000
--------------------------------------------------------------------------------
(In reply to comment #21)
> (Moving back to VERIFIED)
Maciej set the status of this bug to VERIFIED FIXED after it was reopened
through a misunderstanding of the decision process at the a11y teleconference.
Maciej's rationale is that this bug is already escalated to a tracker issue.
VERIFIED with a TrackerIssue keyword is indeed the correct state for a bug that
has been "resolved" by the editor but has not yet in fact been decided by the
working group via the escalation process. Maciej was correct to move the bug
back to VERIFIED per HTML working group process.
http://dev.w3.org/html5/decision-policy/decision-policy.html#states-and-keywords
However, Ian did not FIX this bug. It is not FIXED so VERIFIED FIXED is not
appropriate.
It was last labeled VERIFIED LATER.
http://lists.w3.org/Archives/Public/public-html-a11y/2010Mar/0395.html
Relabeling this bug VERIFIED LATER as that is was last set it to. Maybe a more
appropriate label can be worked out?
================================================================================
#23 Michael Cooper 2011-03-08 16:10:18 +0000
--------------------------------------------------------------------------------
Bug Triage sub-team notes that comment 14 points out that most of the issues
can be addressed. What's missing in this bug is a reference to the track
element that was introduced later. This element resolves the concerns. No
action is required yet as the bug can't be closed until a text format is agreed
upon. Verified Later is an appropriate state at this moment.
================================================================================
--
Configure bugmail: https://www.w3.org/Bugs/Public/userprefs.cgi?tab=email
------- You are receiving this mail because: -------
You are on the CC list for the bug.
Received on Tuesday, 25 September 2012 21:54:25 UTC