Thoughts on accessible web-based screen sharing

Hi all,

I've been on the periphery of a few conversations recently that have touched on the subject of real-time collaboration, screen sharing and meetings. I had some thoughts about how these use cases could be made more accessible, and we thought it might be worth me posing them to you (apologies if any of these have already been covered).

First a brief recap that traditional screen sharing is inaccessible due to sending over a pile of pixels. Back channels have been added to systems like Citrix so that you can have a screen reader running at each end. The remote system can then be explored, because the remote reader has access to its accessibility API. Responsiveness and efficiency is also good, because instead of sending an audio stream from the remote screen reader, essentially only text is being sent over the wire, for the local AT to speak.

We've looked at some situations and specs (such as Viewport Capture [1] and its parent Screen Capture [2]) recently, and wondered if these could be made accessible in a similar way.

At first I thought, if we are in a web context, how about exposing the DOM (or the part of it that's being shared) from the remote viewport as if it were local? That would work much like the remote access approach above, but you wouldn't need a remote screen reader, because the remote DOM is coming to you.

Then I wondered if we could improve on this by building on the Accessibility Object Model (AOM) [3]. As you may know, the AOM is an API for working with the accessibility tree; it allows you to add to and change the accessibility tree without requiring actual corresponding DOM nodes. If using the AOM, we wouldn't even need to transmit a serialisation of the whole DOM; only the relevant subset of it that is the accessibility tree. This would make the whole thing both more efficient and more seamless/robust at the local side.

The two main challenges I can see are that more support for AOM than exists now would probably be needed (maybe this would be a good driver for it), and there could be "containment" issues (like potential duplicate IDs on the local side), though perhaps a simple <iframe> would help there. Of course privacy would be a serious consideration; it's possible that information could be exposed in the accessibility tree beyond that broadcast in the shared screen/viewport video stream. Though if we're talking about using this to access one's own remote system, rather than screen sharing, that wouldn't be a concern.

What do you think?

I'm going to be offline from now until late next week, but wanted to send this off for your consideration. I haven't filed anything in GitHub yet, because it seems way too early, but I'm keen to contribute to any conversations arising from this, if you think it's worth exploring.

best regards,


Matthew

[1] https://www.w3.org/TR/mediacapture-viewport/

[2] https://www.w3.org/TR/screen-capture/

[3] https://wicg.github.io/aom/explainer.html

-- 
Matthew Tylee Atkinson (he/him)
--
Senior Accessibility Engineer
TPG Interactive
https://www.tpgi.com

A Vispero Company
https://www.vispero.com

--
This message is intended to be confidential and may be legally privileged. It is intended solely for the addressee. If you are not the intended recipient, please delete this message from your system and notify us immediately.
Any disclosure, copying, distribution or action taken or omitted to be taken by an unintended recipient in reliance on this message is prohibited and may be unlawful.

Received on Thursday, 26 May 2022 10:16:29 UTC