Re: Call for Consensus (CfC): APA Comment on Media Hints

+1


Lionel Wolberger
COO, UserWay Inc.
lionel@userway.org
UserWay.org <http://userway.org/>
<https://userway.org>[image: text]


On Tue, Jul 13, 2021 at 4:36 PM John Foliot <john@foliot.ca> wrote:

> +1
>
> JF
>
> On Tue, Jul 13, 2021 at 9:19 AM Janina Sajka (janina@rednote.net) <
> janina@rednote.net> wrote:
>
>> +1
>>
>> Janina
>>
>> Janina Sajka writes:
>> > Colleagues:
>> >
>> > This is a Call for Consensus (CfC) to the Accessible Platform
>> > Architectures (APA) Working Group testing whether we have group
>> > consensus on comments and suggested edits on the MediaStreamTrack
>> > Content Hints draft specification, at
>> > https://www.w3.org/TR/mst-content-hint/.
>> >
>> >
>> > <beginning of suggested comment>
>> >
>> > *TITLE *
>> >
>> > APA comments on the MediaStreamTrack Content Hints draft specification,
>> at
>> > https://www.w3.org/TR/mst-content-hint/.
>> >
>> > *OVERVIEW *
>> >
>> > *Content hint attributes defined in this specification will benefit
>> > consumers who rely on assistive technology (AT) and personalization.
>> *The
>> > specification notes its focus on end-users' experience: "Adding a media
>> > -content hint provides a way for a web application to help track
>> consumers
>> > make more informed decision[s]...." Content authors can author
>> contentHint
>> > <
>> https://t.sidekickopen90.com/s3t/c/5/f18dQhb0S7kF8cFFTBW4T_qld2zGCwVN8Jbw_8QsRtKVn1vXj1p1kknW16gGBN41Jd6G101?te=W3R5hFj4cm2zwW4mKLS-4mbkbhW49Ldrl308ybGW4fdgvc41YylgW4fdgXQ41YszVW3H90C_3_SMDQW3zh2Fq3K1LvHW49HR8w1Gy-qYW4fGC1K3R0JW00&si=8000000004174048&pi=58151f60-5af3-4f61-ebc9-364d322a7e5a
>> >
>> >  with the experience of AT users in mind, or UAs acting on behalf of
>> > users.  This specification's introduction would be a good place to
>> > clarify this as a further benefit of content hints. Content authors may
>> > author content hints with AT in mind. In addition, we encourage User
>> Agents
>> > to make this hint available to downstream consumers via API,
>> >
>> > *The specification make no mention of hints regarding support files
>> *(captions,
>> > audio descriptions) that often accompany media content, either linked
>> to it
>> > in HTML externally (using the <track> element) or furnished 'in-band',
>> > e.g., contained within the .MP4 wrapper (HasCaptions: T/F,
>> > HasAudioDescription: T/F). If either return True, THEN they need to be
>> > exposed in the UI: essentially as 'active' buttons in the Controls. Such
>> > support files can be critical to the accessibility of a media track, as
>> for
>> > example when an American Sign Language video is supplied seperately, but
>> > linked. Did the WG consider whether hints could also usefully convey
>> > whether the media content has such supporting files?
>> >
>> > Regarding Section 4: *The specification's hints could address more
>> directly
>> > some common **audio and video formats that are often encountered with
>> > content that has been made accessible. *For clarity, such formats could
>> > propose hints such as these (these are examples for clarity only, we
>> leave
>> > you to define such hints):
>> >
>> > *For Audio, an additional hint to indicate the presence of
>> > audio-description *(or some similar label as you find appropriate).
>> > Audio-description is audio that resembles speech-recognition, but does
>> not
>> > contain data for the purpose of speech recognition by a machine.
>> > Audio-description is audio that resembles "speech" but it will likely
>> not
>> > be appropriate to apply noise suppression or boost intelligibility of
>> the
>> > incoming signal.
>> >
>> > In the language of the specification (4.1) , "A track with content
>> > hint "audio-description"
>> > should be treated as if it contains audio data, without background
>> noise,
>> > describing in words the activity in the video."
>> >
>> >
>> > *For Video, an additional hint to indicate the presence of transcription
>> > embedded in the video*, e.g., motion-with-transcription (or some similar
>> > label as you find appropriate). motion-with-transcription would refer
>> to a
>> > motion video that has, embedded, transcription data, either a
>> > picture-in-picture showing a sign language interpreter, or text captions
>> > embedded in the video.
>> >
>> > In the language of the specification (4.2): A content hint of
>> > motion-with-transcription should be treated such that one region of the
>> > video frame has details that are extra important, and in that region
>> that
>> > significant sharp edges and areas of consistent color can occur
>> frequently
>> > (the area with sign language interpretation, or the area with onscreen
>> > captioned text). This screen region would optimize for detail in the
>> > resulting individual frames rather than smooth playback. Artefacts from
>> > quantization or downscaling should be avoided.
>> >
>> >
>> > *Regarding section 5, the degradation preference does not address
>> regions.*
>> > Picture regions may be very significant for accessibility. Consider a
>> video
>> > with sign language interpretation embedded (e.g., in the upper right
>> > corner), or a video with captions "burned-in" or embedded (e.g., in the
>> > bottom of the picture area). (While APA does not advocate for such
>> embedded
>> > captions, they are common particularly on social media where the default
>> > user behavior is audio "off." These regions would benefit from different
>> > encoding decisions than the rest of the frame.  Regions may be encoded
>> and
>> > decoded quite differently: for example in AVC, "it is also possible to
>> > create truly lossless-coded regions within lossy-coded pictures." *We
>> would
>> > find it useful and supportive of accessible content to make this
>> > information available as an RTCDegradationPreference.*
>> >
>> > Lastly, how are these hints communicated? We note that MP4 files can
>> > contain metadata as defined by the format standard, and in addition, can
>> > contain Extensible Metadata Platform (XMP) metadata. (source::
>> > https://en.wikipedia.org/wiki/MPEG-4_Part_14).
>> >
>> > *REQUESTS*
>> >
>> > Correct these two typos:
>> >
>> >    - Abstract: change "make more informed decision" to either "make a
>> more
>> >    informed decision" or "make more informed decisions"
>> >    - Section 2. change "they appear" to "it appears"
>> >
>> > Add to the introduction that content hint attributes defined in this
>> > specification will benefit consumers who rely on assistive technology
>> (AT)
>> > and personalization.
>> >
>> > The WG to ensure that the specification covers use cases with support
>> > files, and that hints can be provided for those files.
>> >
>> > In section 4, ensure that hints support the use-cases mentioned above.
>> >
>> > In section 5.2 ensure that the specification supports regions
>> particularly
>> > when such regions are important for accessibility.
>> >
>> >    <end suggested comment>
>> >
>> > ***Action to Take***
>> >
>> > This CfC is now open for objection, comment, as well as statements of
>> > support via email. Silence will be interpreted as support, though
>> > messages of support are certainly welcome.
>> >
>> > If you object to this proposed action, or have comments concerning this
>> > proposal, please respond by replying on list to this message no later
>> > than 23:59 (Midnight) Boston Time, Wednesday 14 July.
>> >
>> > NOTE: This Call for Consensus is being conducted in accordance with the
>> > APA Decision Policy published at:
>> >
>> > http://www.w3.org/WAI/APA/decision-policy
>> >
>> > We thank Lionel Wolberger for reviewing this specification on our behalf
>> > and for helping lead our teleconference discussions on this
>> > specification.
>> >
>> > Janina and Becky
>> >
>> > --
>> >
>> > Janina Sajka
>> >
>> > Linux Foundation Fellow
>> > Executive Chair, Accessibility Workgroup:     http://a11y.org
>> >
>> > The World Wide Web Consortium (W3C), Web Accessibility Initiative (WAI)
>> > Chair, Accessible Platform Architectures      http://www.w3.org/wai/apa
>> >
>>
>> --
>>
>> Janina Sajka
>> https://linkedin.com/in/jsajka
>>
>> Linux Foundation Fellow
>> Executive Chair, Accessibility Workgroup:       http://a11y.org
>>
>> The World Wide Web Consortium (W3C), Web Accessibility Initiative (WAI)
>> Co-Chair, Accessible Platform Architectures     http://www.w3.org/wai/apa
>>
>>
>>
>
> --
> *John Foliot* | Senior Industry Specialist, Digital Accessibility
>
> "I made this so long because I did not have time to make it shorter." -
> Pascal "links go places, buttons do things"
>