Text description for @poster (was RE: CP, ISSUE-30: Link longdesc to role of img [Was: hypothetical question on longdesc])

Silvia Pfeiffer wrote:
> Yes, I appreciate that. I don't think there is any dispute of that.
> I'm just saying that since we don't know what frame the author *or
> browser* chooses to represent the video - of the thousands of frames
> that a video contains (or in fact any image) - the frame is only
> that:
> a frame representing a short glimpse of the video.

*IF* a) the author chooses to select a frame of that video, convert it to a
.jpg and then reference that .jpg via @poster, or b) the poster frame is not
author declared, at which point the browser is free (per the spec) to select
a frame from the video to display as the placeholder image, *THEN* your
assertion is correct. 

However if the author chooses to declare that placeholder image using
@poster, then we *DO* know what the frame (or any other graphic image) is -
as it is being expressly declared via @poster, so you cannot always say we
don't know what frame the author is choosing.

What you keep sidestepping however is the scenario whereby the art
department creates an image outside of the film director's series of moving
images. In this case, the page author will be referencing an image that
never originated from the film itself, but from a separate, disparate
source. This has not been addressed.


Let's dissect the spec. It states:
	"The video element is a media element whose media data is ostensibly
video data, possibly with associated audio data."

Question: is a JPG video or audio data?

The spec continues:
	"The poster attribute gives the address of an image file that the
user agent can show while no video data is available. The attribute, if
present, must contain a valid non-empty URL potentially surrounded by
spaces."

Confirmation that the @poster {object} is in fact an image file, and not
video data. Further, also a backhanded acknowledgement that it is a separate
file that *isn't* a video.

So physically they are different data objects, different data types. Since
both the Video (mime-type = video/mp4) and the image (mime-type =
image/jpeg) are both visual assets, they both require textual equivalents.

Leonie, Janina, Everett Zuefelt (who first encouraged the filing of the
original bug), and others, want descriptions of both. They deserves them,
and we *can* figure out how to do this, but first we need to agree that
there are two {objects} here that require longer textual descriptions.
 

> Therefore, there is
> no semantic difference between what the video contains and what the
> image contains: you can in fact identify the video with the long
> textual description and the image with the short textual description.

Respectfully - No. There is a semantic difference, and there is a physical
difference. What you are suggesting (at least, what I am hearing) is that
conceptually to you, a sighted person, there is a symbiotic and logical
linking of the two objects that create a composite understanding of what it
is you are looking at being physically rendered on screen.

> Now, my solution for the short textual description of a video was
> @aria-label. If that is indeed not possible (like John seems to
> argue,

Be careful of paraphrasing what I am saying.  Silvia, a significant part of
this is understanding what the various values in the Accessibility APIs
take, what they do, and how they "work".

In the Accessibility APIs, they have the concept of an AccName (Accessible
Name). The ARIA Implementation Guide[1] explains how to determine the
Accessible Name this way[2] (paraphrased):

Both aria-labelledby and aria-label map to the AccName. User agents are
supposed to first look for an aria-labelledby value; if none exists it falls
back to looking for an aria-label. If neither exists, then the instruction
is to look for a host-language equivalent (in HTML that equates to @alt),
and if *that* does exist, then look for a unique identifier (such as a file
name).  

This can be "proven" with standard images today: screen readers "finds" an
image and looks first for an aria-labelledby value, then an aria-label
value, then an @alt value, and then something unique. We rarely see the
first 2 in garden-variety HTML docs, but often see @alt (good!). In that
case, the screen reader uses, as the AccName, the string value of @alt: if
that value is {null} (aka alt="") then the reader "reads" the null value and
says nothing (it is also, at this point, the equivalent to
role="presentation", which will also have the same effect). However, when
there is no alt at all, not even alt="", then it drills down further (as
described above) and finds something unique - the file name - and announces
that out loud.

So, to the AAPIs, essentially aria-labelledby = aria-label = @alt. 

However, while aria-labelledby *can* express rich text (as it is marked up
and in the viewport), both aria-label and @alt can only take string text,
which means both aria-label and @alt cannot render HTML rich content, such
as alt="<span lang="fr">Le petit d&eacute;jeuner</span> - a short story".
This is true of @alt, and this is true of aria-label.

However, if this i18n (internationalization) support is not deemed
important, then sure, aria-label (or @alt) could be used for the AccName
(short-text), but so could aria-labelledby (with the added bonus that it
*could* be HTML rich). What I have previously asserted is that the i18n
requirement is not one we should be dismissing.

  [1]
http://www.w3.org/WAI/PF/aria-implementation/#mapping_state-property_table 
  [2] http://www.w3.org/WAI/PF/aria-implementation/#mapping_additional_nd_te


*********

Now, let's define what we have, and what we need. We have two visual assets,
and each asset requires both a short textual description/name, and a longer
textual description. In practice (and in a nod to your perception of the two
assets) they both could likely share the same name/description (AccName).

Best:
	<h2 id="movie_title"><span lang="fr">Le petit d&eacute;jeuner</span>
- a short story</h2>

	<video aria-labelledby="movie_title">


Good (but poor i18n support):
	<video aria-label="Le petit dejeuner a short story">

Possibly also Good (but again poor i18n support):
	<video alt="Le petit dejeuner a short story">


Bad:
	<video>

Short text/ AccName resolved (sorta).


*********

Next, we need 2 longer textual descriptions. The need for structured HTML
rich text here is significantly more important than above (AccName), and to
get screen readers to express that rich marked-up text it must be part of
the viewport. 

The oldest mechanism we have that delivers that functionality is @longdesc
(oh yes, after all, it's still in the subject line) - where, *when
supported* it presents a link to the user, who follows that link, GETS the
new content, renders in the view port (reads it aloud), and then the user
disposes of that page content (via back-button, history.back(), etc.). If
you review Rich and Steve's Unofficial Draft for aria-describedat, this is
exactly what that new aria-attribute does too.

So, we *could* use either @longdesc *OR* aria-describedat to provide at
least one longer textual description, (if we had @longdesc in HTML5, but
well, you know... and aria-describedat is currently too immature to
realistically consider today) but for what? As an attribute, it will be
applied to the <video> element, which is, by definition of the spec "a media
element whose media data is ostensibly video data".  Thus I would suggest
that the longer textual description attribute "thing" would be indirectly
associated to the video data associated to the <video> element.

This still leave us with a need to find a mechanism to associate the longer
textual description to the other {object} - the JPG file.


Meanwhile, Charles Pritchard wrote:
>
> The concept here is that authors are going to use <video poster> as
> shorthand for <div><img onclick="this.display='none';
> this.nextSibling.display='inline';" /><video /></div>
  <snip>
> There are a lot of opportunities to describe this within the video's
> label or description. There most certainly are.
> But there are times when this falls short. That's because <video
> poster>
> is a composite element.
> 
> It means <video><img /></video>

For the longest time, I too thought of it in such a fashion, until one day I
thought of it in a slightly different way. What if, *conceptually*, we think
of the @poster image as something more equivalent to a CSS background image?

  <video src="file.webm" style="background-image: url(poster.jpg);">

When the video isn't playing, it doesn't render (style="opacity:0;"), and
thus the "background image" shows through: once the video start to play,
then it sits "on top off" the background image (style="opacity: 1;"),
obscuring the image. (Please remember that this is a conceptual model at
this time, and not the mechanical model).

But this got me to thinking (and talking) and it occurred to me that there
may be other times when having the ability to apply a textual description to
a background image might actually exist. I have some ideas there, but it is
late, this is already too long, and/but I need to write them up. However,
*if* it makes sense, then we could create a new mechanism (attribute) that
would allow us to link a longer textual description to a back-ground image
(or @poster image) - in other words a multi-purpose attribute that would
solve this problem.

I have approached the ARIA-WG with this idea, and it has not been rejected:
in fact it is on the same short list as aria-describedat is on (the ARIA 1.1
working list). 

This is still a very nascent idea at this time, but it is one I am extremely
hopeful will see the light of day. 


Irrespective of all of this, we also need to find a method of linking the
full text transcript of the video (Issue 195). This is *NOT* a description
of the video (nor the poster) but a different kind of text file that is also
related to the video itself.

I have, at this time, partially given up on trying to get the HTML WG to
break through and understand all of this: I have tried in the past to
advocate for a special element that would allow for a longer textual
description for the JPG {object} (aka Issue 142 - Status Formal Objection)
and Issue 204 is currently being rejected by the Chairs, as what I submitted
('we're stalled because we need answers on other Issues first') is something
they deem not to be responsible for.  If HTML5 does not have the full set of
requirements met as articulated in Issue 204, I'll file a Formal Objection,
and their inability to equitably manage process will be one of the grounds
for the filing (but I digress...)


At any rate, I remain hopeful that we will find a solution inside of ARIA
for the ability to provide a longer textual description for the video
poster. I suspect that at this time it is too late to get something into
HTML5 as a native attribute, but if there is a way, I would be open to
working towards that too.

JF

Received on Saturday, 24 March 2012 07:43:53 UTC