Re: Clarification on the notion of "Second Screen"?

Good morning Francois,

First, my bad, I was supposed to follow up on this with you last week.
Allow me to rephrase the question if I may:

We understand the first use-case, where accessibility support files are
explicitly referenced in the HTMl via the source and track attributes.
However, because of mobile latency issues and the overhead required in
ensuring two seperate files remain in sync, content authors can
*alternatively* provide the caption files (etc.) *inside* of the MP4
wrapper, and (as we understand it) an API extracts those piece-parts at the
user-agent end.

The net effect is that in that scenario the hosting page is now only
sending *one file* down the wire, as opposed to 2 or more (i.e.
src="foo.mp4" track="bar.ttml" when referenced separately in HTML)

And so, we just want to ensure that in the use-case where the support files
are bound inside of the mp4 file, that the second screen will still be able
to extract that data - and we envision sometimes ONLY the second screen
exposing that data to the end user. If this is the case then we are good to
go.

Thanks.

JF

On Tue, Jan 11, 2022 at 9:45 AM Francois Daoust <fd@w3.org> wrote:

> Hello Janina,
>
> Thanks and happy new year too!
>
> I assume that you're talking about the source and track elements in
> HTML, which allow to specify alternative media resources and point to
> out of band captions and other tracks.
>
> I confirm that these elements are supported by the Remote Playback API
> and the Open Screen Protocol: the URLs of source and track elements will
> be communicated to the receiver so that it may retrieve them and provide
> the alternative content.
>
> Francois.
>
> ------ Original message ------
> From: "Janina Sajka (janina@rednote.net)" <janina@rednote.net>
> To: "Francois Daoust" <fd@w3.org>
> Cc: "W3C WAI Accessible Platform Architectures" <public-apa@w3.org>
> Date: 10/01/2022 15:42:16
>
> >Hello, Francois, and Happy New Year!
> >
> >My apologies for the month of silence on our concerns. The good news is
> >that APA did discuss our concerns and your response during our calls on
> >the 15th of December and also last week on the 5th of January.
> >
> >We're generally happy with your response, but have one additional point
> >to clarify.
> >
> >During the development of HTML 5 we were insistant on a use case where
> >the alternative media--captions, descriptions of video, etc--might be
> >inband or out of band, meaning that they might be packaged together in
> >an mp4 or similar, or might be provided by a third party separately. Our
> >use case was a university service making some resource accessible to
> >students with particular alternative media needs. The mechanisms to
> >support this did make HTML 5.
> >
> >APA would like to clarify that your very specific response will apply
> >regardless?
> >
> >I believe this is our only remaining concern, and I thank you for your
> >thorough response below.
> >
> >Best,
> >
> >Janina
> >
> >Francois Daoust writes:
> >>  Dear Janina,
> >>
> >>  Thanks for the response.
> >>
> >>  Your question applies to the Remote Playback API and the Open Screen
> >>  Protocol on top of which the Remote Playback API may be built. I
> confirm
> >>  that the Open Screen Protocol has all the necessary features to
> communicate
> >>  all the alternative media. I also confirm that the Remote Playback API
> asks
> >>  user agents to synchronize the media element state (the set of all
> single
> >>  media element properties observable by the page and/or the user)
> between the
> >>  controller and the receiver.
> >>
> >>  As such, I believe that the API handles all of the concerns that you
> raise.
> >>  Captions and subtitles will be present along with the main audio/video
> >>  content. That includes audio or text descriptions of video, provided
> that
> >>  these descriptions are represented as audio and text tracks (in other
> words
> >>  that they are part of the media element state, and not some DOM content
> >>  available elsewhere on the page).
> >>
> >>  I would just note that the spec leaves the implementation of the
> >>  synchronization step up to the controller and receiver (the Open Screen
> >>  Protocol defines one way to do it but there is no requirement that the
> Open
> >>  Screen Protocol be supported), and that it does not take for granted
> that
> >>  the receiver supports the exact same set of capabilities as the
> controller.
> >>
> >>  Thanks,
> >>  Francois.
> >>
> >>
> >>  ------ Original message ------
> >>  From: "Janina Sajka (janina@rednote.net)" <janina@rednote.net>
> >>  To: "Francois Daoust" <fd@w3.org>
> >>  Cc: "Michael Cooper" <cooper@w3.org>; "APA Chairs" <
> group-apa-chairs@w3.org>
> >>  Date: 14/12/2021 14:24:30
> >>
> >>  > Dear Francois:
> >>  >
> >>  > First, my apologies for my silence over the past week. I'm not
> intending
> >>  > to block your charter unnecessarily.
> >>  >
> >>  > So, let me ask my question a different way. Let's forget about
> braille
> >>  > displays (or any particular second screen device). We know there are
> a
> >>  > range of devices, some smart enough to serve as second screen
> devices,
> >>  > others just what we used to call "dumb terminals."
> >>  >
> >>  > The core concern for APA is whether all the alternative media is
> being
> >>  > communicated via the API you're proposing. Can we count on the API
> for
> >>  > captions? As many different languages of captions and subtitles as
> might
> >>  > be available? Also for descriptions of video, whether audio or text
> >>  > descriptions of video?
> >>  >
> >>  > If all of that is facilitated, then I think APA can sign off. Does
> this
> >>  > restatement of our concern help clarify?
> >>  >
> >>  > Best,
> >>  >
> >>  > Janina
> >>  >
> >>  > Francois Daoust writes:
> >>  > >  Hi Janina,
> >>  > >
> >>  > >  I confirm that Bluetooth headset, and speakers, are included in
> the
> >>  > >  definition of "screen" (the charter rather uses the term
> "presentation
> >>  > >  display"). In particular, the draft charter explicitly mentions
> Bluetooth
> >>  > >  and includes the following sentence: "For the purposes of this
> charter,
> >>  > >  presentation displays include wireless speakers as well".
> >>  > >
> >>  > >  That said, I am not sure whether the group considers refreshable
> braille
> >>  > >  displays to be a possible "presentation display" as well. My
> limited
> >>  > >  understanding of refreshable braille displays is that they are
> meant to
> >>  > >  display characters, whereas the presentation displays being
> considered by
> >>  > >  the Second Screen Working Group are either those capable of
> rendering audio
> >>  > >  or video streams, or those capable of running HTML applications.
> >>  > >
> >>  > >  Said differently, the APIs developed by the group may be used to
> stream
> >>  > >  audio and/or video content to a presentation display, or to
> establish a
> >>  > >  communication channel between a web application running on a
> controller
> >>  > >  device and a web application running on the presentation display.
> However,
> >>  > >  there are no provisions to stream pure text content to a
> presentation
> >>  > >  display.
> >>  > >
> >>  > >  Or do I misunderstand what refreshable braille displays encompass?
> >>  > >
> >>  > >  Thanks,
> >>  > >  Francois.
> >>  > >
> >>  > >
> >>  > >  ------ Original message ------
> >>  > >  From: "Janina Sajka (janina@rednote.net)" <janina@rednote.net>
> >>  > >  To: "Francois Daoust" <fd@w3.org>
> >>  > >  Cc: "Michael Cooper" <cooper@w3.org>
> >>  > >  Date: 07/12/2021 15:48:32
> >>  > >
> >>  > >  > Hi, Francois:
> >>  > >  >
> >>  > >  > My apologies for being slow to follow up on this. Our main
> concern is to
> >>  > >  > continue to capture what we agreed some years back, perhaps
> 2015-2016?,
> >>  > >  > that "screen" should be understood to be generic, meaning that
> the
> >>  > >  > screen might be a bluetooth headset, or speaker pair, or a
> refreshable
> >>  > >  > braille display.
> >>  > >  >
> >>  > >  > This understanding grew out of our work on media accessibility
> in HTML
> >>  > >  > 5, and we did have an exchange at some point years ago
> clarifying this
> >>  > >  > understanding. Our issue is really just that, and nothing more.
> We did
> >>  > >  > thing it worth clarifying because several years have passed,
> and it's
> >>  > >  > likely many participants in Second Screen may be newer than
> that old
> >>  > >  > exchange of understanding.
> >>  > >  >
> >>  > >  > I will be in other meetings for the next few hours, but will
> get back to
> >>  > >  > you with more later today. Again, sorry for holding things up!
> >>  > >  >
> >>  > >  > Janina
> >>  > >  >
> >>  > >  > Francois Daoust writes:
> >>  > >  > >  Hi Janina,
> >>  > >  > >
> >>  > >  > >  Michael noted that APA reviewed the proposed Second Screen
> Working Group
> >>  > >  > >  charter and would like a clarification about the meaning of
> the term "second
> >>  > >  > >  screen". Could you provide more detail as to what APA is
> wondering about?
> >>  > >  > >
> >>  > >  > >  The proposed charter is at:
> >>  > >  > > https://w3c.github.io/secondscreen-charter/
> >>  > >  > >
> >>  > >  > >  Thanks,
> >>  > >  > >  Francois, staff contact for the Second Screen Working Group.
> >>  > >  >
> >>  > >  > --
> >>  > >  >
> >>  > >  > Janina Sajka
> >>  > >  > https://linkedin.com/in/jsajka
> >>  > >  >
> >>  > >  > Linux Foundation Fellow
> >>  > >  > Executive Chair, Accessibility Workgroup:    http://a11y.org
> >>  > >  >
> >>  > >  > The World Wide Web Consortium (W3C), Web Accessibility
> Initiative (WAI)
> >>  > >  > Co-Chair, Accessible Platform Architectures
> http://www.w3.org/wai/apa
> >>  > >  >
> >>  >
> >>  > --
> >>  >
> >>  > Janina Sajka
> >>  > (she/her/hers)
> >>  > https://linkedin.com/in/jsajka
> >>  >
> >>  > Linux Foundation Fellow
> >>  > Executive Chair, Accessibility Workgroup: http://a11y.org
> >>  >
> >>  > The World Wide Web Consortium (W3C), Web Accessibility Initiative
> (WAI)
> >>  > Co-Chair, Accessible Platform Architectures
> http://www.w3.org/wai/apa
> >>  >
> >
> >--
> >
> >Janina Sajka
> >(she/her/hers)
> >https://linkedin.com/in/jsajka
> >
> >Linux Foundation Fellow
> >Executive Chair, Accessibility Workgroup:      http://a11y.org
> >
> >The World Wide Web Consortium (W3C), Web Accessibility Initiative (WAI)
> >Co-Chair, Accessible Platform Architectures    http://www.w3.org/wai/apa
> >
>
>
>

-- 
*John Foliot* |
Senior Industry Specialist, Digital Accessibility |
W3C Accessibility Standards Contributor |

"I made this so long because I did not have time to make it shorter." -
Pascal "links go places, buttons do things"