W3C home > Mailing lists > Public > public-vocabs@w3.org > September 2013

Re: Schema.org accessibility proposal Review...

From: Richard Schwerdtfeger <schwer@us.ibm.com>
Date: Thu, 5 Sep 2013 15:08:42 -0500
To: Gerardo Capiel <gerardoc@benetech.org>
Cc: "a11y-metadata-project@googlegroups.com" <a11y-metadata-project@googlegroups.com>, Alexander Shubin <ajax@yandex-team.ru>, Charles McCathie Nevile <chaals@yandex-team.ru>, Charles Myers <charlesm@benetech.org>, Dan Brickley <danbri@google.com>, Egor Antonov <elderos@yandex-team.ru>, Jason Johnson <jasjoh@microsoft.com>, George Kerscher <kerscher@montana.com>, Liddy Nevile <liddy@sunriseresearch.org>, Matt Garrish <matt.garrish@bell.net>, "<public-vocabs@w3.org>" <public-vocabs@w3.org>
Message-ID: <OFAC97C849.DF25EFBD-ON86257BDD.006DDAF1-86257BDD.006EA8C8@us.ibm.com>

Rich Schwerdtfeger

Gerardo Capiel <gerardoc@benetech.org> wrote on 09/05/2013 12:52:21 PM:

> From: Gerardo Capiel <gerardoc@benetech.org>
> To: Charles McCathie Nevile <chaals@yandex-team.ru>, "a11y-metadata-
> project@googlegroups.com" <a11y-metadata-project@googlegroups.com>,
> "<public-vocabs@w3.org>" <public-vocabs@w3.org>,
> Cc: Dan Brickley <danbri@google.com>, Alexander Shubin <ajax@yandex-
> team.ru>, Egor Antonov <elderos@yandex-team.ru>, Liddy Nevile
> <liddy@sunriseresearch.org>, Charles Myers <charlesm@benetech.org>,
> Richard Schwerdtfeger/Austin/IBM@IBMUS, George Kerscher
> <kerscher@montana.com>, Jason Johnson <jasjoh@microsoft.com>, Matt
> Garrish <matt.garrish@bell.net>
> Date: 09/05/2013 12:53 PM
> Subject: Re: Schema.org accessibility proposal Review...
> Chaals,
> Thank you for taking the time to think through these issues.  Responses
> Gerardo Capiel
> VP of Engineering, benetech
> Twitter: @gcapiel  Fork, Code, Do Social Good:

> On Sep 4, 2013, at 5:57 PM, Charles McCathie Nevile
>  wrote:
> Hi,
> I'm really sorry this has taken me so long to pin down. I appreciate
> that Epub are looking for some fast movement from schema.org at the
> moment, but I am wondering if we can provide that (I actually
> believe it is possible to fix the things I talk about here without
> busting that timeline, but I can't guarantee that).
> I'd like to discuss this on a publicly archived email list. I'm not
> sure if ti should be public-vocabs, or an accessibility-oriented one, or
> I've added the public accessibility metadata project email list and
> public-vocabs list to this thread.
> I finally figured out what made me uneasy about the proposal - until
> now I just felt like the dodgy examples I had seen were a symptom
> rather than just one-off mistakes. I've described the issues below as
> - AccessMode (and as a sub-issue, levels of detail). I think this is
> a serious thing that should be fixed more or less immediately
> - AccessFeature. This is important enough to look at immediately and
> figure out what it really does.
> - adaptations. We should let others define this. We really only need
> "anotherVersion" pointers in most cases

> - ATCompatible. I think this is broken, but not critically
> important. We should fix it when we've done the above stuff
Actually, it was supposed to be ATInteroperable (from Access For All). It
means that the web content was designed to be interoperable with an
assistive technology and the original spec. states which WCAG checkpoints
are applicable. This is extremely important. By interoperable it means that
the content is capable of enabling a browser to support the native platform
accessibility APIs. For HTML5 this would mean through the use of content
with strong native semantics and/or ARIA support. ATInteroperable does not
mean fully WCAG compliant but it adheres to a subset of the requirements.
For example, A Screen Reader, Screen Magnifier, or accessibility test tool
user would ask for ATInteroperable content. This type of vocabulary is also
sucha that it does not declare the users medical disability.

ATCompatible is the wrong choice as it could mean that your keyboard keys
don't conflict by looking at the name.

> - evolution. We need to have some idea about how to handle this, but
> a formal specification isn't required IMHO.
> = accessMode =
> It should be possible for a "single resource" to be available with
> more than one *set* of accessModes.
Yes, we should have a unique set of access modes. Access For All defined
those. I suspect they made it to schema.org.

> I agree and this is the design.  A single resource can require one
> or more accessMode(s).
> Per IMS documentation at the links below, the accessMode property
> describes "Human sensory perceptual system or cognitive faculty
> through which a person may process or perceive information."  The
> links below also provide example use cases:
> http://www.imsglobal.org/accessibility/afav3p0pd/

> AfA3p0_DESinfoModel_v1p0pd.html#_Toc323719827
> http://www.imsglobal.org/accessibility/afav3p0pd/

> AfA3p0_BestPractice_v1p0pd.html#_Toc324315328
> We have also published a best practices and an implementation guide
> on the use of accessMode at:
> http://www.a11ymetadata.org/wp-content/uploads/2013/04/

> A11yMetadataProjectBestPracticesGuide_V.6.pdf
> https://wiki.benetech.org/display/a11ymetadata/Practical+Properties

+Guide .
> All of our resources for assistance and guidance can be found at
> http://www.a11ymetadata.org/resources/ .
> accessMode is derived from the Access For All (AfA) specification by
> IMS Global and the associated ISO 24751 (Individualized adaptability
> and accessibility in e-learning, education and training) Part 3
> (digital resource description) specification. WGBH’s NCAM and the
> Inclusive Design Research Center (IDRC) who were both involved in
> ISO 24751 and AfA were also involved in the crafting of this Schema.org
> accessibility metadata proposal.  IMS who is the copyright holder on
> AfA released this proposal under a CC-BY-SA license per Schema.org terms.
> We chose to leverage this prior work because Schema.org is meant to
> address search use cases, which have strong overlap with the AfA use
> cases related to matching the appropriate resource to a learner
> based on on their needs and preferences.  AfA/ISO 24751 work is
> being synchronized with and leveraged by other current projects,
> such as Indie UI and GPII.  Rich Schwerdtfeger can probably talk to
> this much more than I can.
> Essentially, it is unclear what accessMode really means. At least
> for the search use case where a user needs to find a suitable
> resource, I think the only way this can really work is if an
> accessMode is a set of one or more *required user capabilities*.
> Yes, I think that is what we are saying too.  In fact, it seems like
> what others have understood on the public vocabs list as evidenced by:
> http://lists.w3.org/Archives/Public/public-vocabs/2013May/0092.html

> http://lists.w3.org/Archives/Public/public-schemabibex/2013Aug/0049.html

> To pick some examples:
> Alice In Wonderland is available as a resource which only *requires*
> the ability to read text.
> We happen to have a few versions of Alice in Wonderland in Bookshare
> that have been indexed by Google's Custom Search Engine, so let's
> first search for a pure text version of it with no visual components:
> http://www.google.com/cse?

> cx=001043429226464649088:WMX1711548652&q=more:p:book-
> accessmode:textual%20-more:p:book-accessmode:visual%20more:p:book-
> name:alice%20in%20wonderland
> If you inspect one of those titles using Yandex's webmaster tools or
> Google's rich snippets tool that lets me just plug in a URL
> parameter, we can see the accessibility microdata on that page:
> http://www.google.com/webmasters/tools/richsnippets?q=https%3A%2F%

> 2Fwww.bookshare.org%2Fbrowse%2Fbook%2F18041
> As you can see we offer 4 different adaptations of that title, which
> we originally adapted from a physical book that we scanned.  For
> each adaptation we describe the accessModes that are used.
> If I wanted to search for Alice in Wonderland as an audio book in
> Bookshare we can do that and find that same title, since we offer
> audio only adaptations on that page, such as DAISY Audio and MP3:
> http://www.google.com/cse?

> cx=001043429226464649088:WMX1711548652&q=more:p:book-
> accessmode:auditory%20more:p:book-name:alice%20in%20wonderland
> Here’s another example that searches for books about history in
> Bookshare where the images are either described with short (i.e.,
> alt) or long descriptions (i.e., longdesc or DAISY prodnote):
> http://www.google.com/cse?

> cx=001043429226464649088:WMX1711548652&q=history%20more:p:book-
> mediafeature:alternativetext,longdescription

> This applies to several versions available from Project Gutenberg -
> some of them have images, some don't, but in any case it is possible
> to understand Alice in Wonderland without the pictures.
> This is arguably not the case with kipling's "Just So Stories"
> (likewise freely available), since there are intrinsic references to
> the images - although it might be the case that the references are
> sufficiently descriptive
> If there is a movie of Alice In Wonderland available with captions,
> then there are two sets of requirements. One is to be able to watch
> a movie and listen to the audio, and the second set is to be able to
> watch a movie and read the captions. (If it had descriptive audio, a
> third would be being able to hear the descriptive audio and the main
> That is the purpose of mediaFeature.  A movie typically has
> accessModes of visual and auditory.  If it has captions, we simply
> add a mediaFeature of captions and/or a mediaFeature of
> audioDescription.  You can find an example in our Practical Properties
> https://wiki.benetech.org/display/a11ymetadata/Practical+Properties

> +Guide#PracticalPropertiesGuide-Video
> Khan Academy has been collaborating with us on implementing this for
> their videos and Gooru Learning’s search engine will be indexing
> this metadata to offer their users the ability to filter search
> results to only videos that are captioned.  We hope that Google will
> fix a “bug” with their current video search, which is demonstrated
> by the below search on Wikimedia Commons for captioned videos of the
> moon landing:
> http://goo.gl/7PTmI

> Though I checked and found moon landing videos on the Commons site
> that are captioned, Google only displays results for videos that are
> captioned from their own YouTube property for which they have that
> information.

> It might be that this assumption is already made in the proposal,
> but it is not explicit, and the proposal doesn't explain how to deal
> with a single version that can be used with different sets of
> capabilities (whereasit includes a way to point to other versions)
> Schema.org allows for multiple values for the same property.  For
> example, take a look at the example properties for ingredients at
> the bottom of this page: http://schema.org/Recipe

> == details in accessMode ==
> the top-level accessMode is only useful for "most" cases. Knowing
> that I am physically capable of watching and listening to a video is
> great, unless it is only available in a format I cannot use such as
> "flash for my iPhone 3", "ogg theora/vorbis"….
> Schema.org already has a properties for formats (e.g., http://
> schema.org/encodingFormat , http://schema.org/BookFormatType )

> Similarly if I have more detailed requirements. For example although
> I can watch and hear a video tutorial, being red-green colourblind
> or requiring very high contrast may mean I can't actually understand
> it in practice.
> For those use cases, we have accessMode = colorDependent and mediaFeature
> = accessFeature =
> It isn't clear what the requirements and use cases are for accessFeature.
> Are these just cataloguing information, similar to noting that a
> physical book has 5 pages of color photographs, or a guarantee that
> e.g. each image has alternative text, or a longer description is
> provided as necessary?
> We thought that implementing AfA or ISO 24751 verbatim for the
> purposes of web search and for use by web masters would be too
> complicated: it needed to be a small set of properties that a user
> could deal with. We decided to combine various ideas into
> mediaFeature.  mediaFeature simply states that the author/publisher
> or someone who modified that book, tried to take advantage the
> stated accessibility features.  For example we make no promise that
> every image was described appropriately, since it is hard to define
> that by a single person and actually different consumers of the
> information may have different opinions about the quality and
> coverage of the image descriptions.

> It seems that they are the former - interesting information that
> might lead to expectations of the resource described. They don't
> seem well-enough specified to know if I can rely on them. And the
> list seems somewhat arbitrary.
> It isn't clear, for example, how to classify a resource that assumes
> the ability to touch several points of a screen at once, which is
> now a common feature on devices but poses an obvious potential
> problem to some users such as those who don't have very many fingers.
> In this use case, the resource that needs to be described is an
> application, not the content.  If the accessibility API of this
> application supports multi-touch with the user’s AT, then the
> proposed ATCompatible and accessAPI properties should be relevant.
> Furthermore, we also have a property called controlFlexibility,
> which also should provide information about whether the application
> can be fully controlled either via audio, mouse, keyboard, touch or
> video/gestures.
> = adaptations, other versions, ... =
> Liddy has pointed out that the concept of "the original" is only
> sometimes interesting, and that it isn't an accessibility concept
> but a more general one related to creative works.
> Actually to a teacher, the “original” is important if they are
> trying to find a resource for a disabled student that is an
> adaptation of the “original” resource used by the rest of the
> students in the classroom who don’t have a disability.  Similarly a
> publisher or book seller wants to make it easy for consumers with
> specialized needs to find resources that have been adapted or “made
> accessible.”  These adaptations properties are also derived from AfA
> and ISO 24751 and I think it’s worthwhile reading their
> specifications to understand  their reasoning. We did simplify the
> terms (removing partial adaptation and the like) to simplify this
> for search purposes.

> I think it sometimes matters and sometimes doesn't ("born-digital"
> works don't necessarily have a canonical original form, but may be
> served in very different forms to different users).
> I think we should generally not try to coin accessibility-specific
> ways of talking about different versions of acreative work. FRBR and
> its many analogous efforts already deal with this issue, and we
> should just use whatever schema.org adopts where we need it. A
> generic "anotherVersion" relation is probably the most commonly
> required thing anyway.
> This isn't high priority, except that I think we should quickly
> decide we want to avoid dealing with defining stuff that others have
> spent decades on, and instead try to use their work...
> = ATCompatible =
> being a boolean seems a bit naive. I can think of lots of resources
> that work with specific assistive technologies, but fail with others
> (my instant example is http://cookiecrook.com/longdesc/iframe/ which
> works great on VoiceOver but as far as I can tell does nothing to
> work effectively with the built-in MacOS magnifier - two assistive
> technologies I have readily available…).
> In the case of content, we once again, we tried to simplify the work
> for web masters and searchers.  Since we can always expand the
> vocabulary, we decided to err on simplicity, and to just have the
> focus on whether accessible technology was considered; enumerating
> WCAG sections would be overly daunting in a search. In the case of
> applications, we did provide greater granularity with the accessAPI
> property, which would differentiate what accessibility API, such as
> VoiceOver, the application is compatible with.

> I don't think ATCompatible is the first priority for fixing. But I
> do think it should be replaced with something because I don't think
> it is very useful.
> = Evolution =
> Who looks after the terms, and how do they get extended?
> This is probably the problem with any Schema.org proposal.  By
> basing this proposal on existing standards, such as AfA and ISO
> 24751, it will be easier to evolve these terms as more people and
> organizations (e.g., IMS, GPII, IDPF) will be invested in keeping
> these terms synchronized with other standards and technologies. We
> believe that there are some efforts with 24751 to have a registry to
> assist in this process.

> Some things are pretty basic and could be reliably laid down - the
> ability to see or hear is something that we more or less understand.
> But others, such as interacting with new types of device, will
> probably change. Pointer and especially touch interfaces have
> changed a lot in this century, voice interfaces are starting to do
> so, and gestural things like Wii and Kinect are beginning to appear
> more often too.
> That's enough comment for now. Hopefully I've clarified the main
> threads that I think need action (some more urgently than others).
> cheers
> Chaals
> --
> Charles McCathie Nevile - Consultant (web standards) CTO Office, Yandex
>      chaals@yandex-team.ru         Find more at http://yandex.com
Received on Thursday, 5 September 2013 20:09:36 UTC

This archive was generated by hypermail 2.4.0 : Friday, 17 January 2020 17:49:07 UTC