W3C home > Mailing lists > Public > public-vocabs@w3.org > September 2013

Re: Schema.org accessibility proposal Review...

From: Gerardo Capiel <gerardoc@benetech.org>
Date: Thu, 5 Sep 2013 17:52:21 +0000
To: Charles McCathie Nevile <chaals@yandex-team.ru>, "a11y-metadata-project@googlegroups.com" <a11y-metadata-project@googlegroups.com>, "<public-vocabs@w3.org>" <public-vocabs@w3.org>
CC: Dan Brickley <danbri@google.com>, Alexander Shubin <ajax@yandex-team.ru>, Egor Antonov <elderos@yandex-team.ru>, Liddy Nevile <liddy@sunriseresearch.org>, Charles Myers <charlesm@benetech.org>, Richard Schwerdtfeger <schwer@us.ibm.com>, George Kerscher <kerscher@montana.com>, Jason Johnson <jasjoh@microsoft.com>, Matt Garrish <matt.garrish@bell.net>
Message-ID: <2CECD851-F8A1-403E-B000-EDF3E45DC826@Benetech.org>

Thank you for taking the time to think through these issues.  Responses below:

Gerardo Capiel
VP of Engineering, benetech
Twitter: @gcapiel<http://twitter.com/gcapiel>  Fork, Code, Do Social Good: http://benetech.github.com/

On Sep 4, 2013, at 5:57 PM, Charles McCathie Nevile <chaals@yandex-team.ru<mailto:chaals@yandex-team.ru>>


I'm really sorry this has taken me so long to pin down. I appreciate that Epub are looking for some fast movement from schema.org<http://schema.org> at the moment, but I am wondering if we can provide that (I actually believe it is possible to fix the things I talk about here without busting that timeline, but I can't guarantee that).

I'd like to discuss this on a publicly archived email list. I'm not sure if ti should be public-vocabs, or an accessibility-oriented one, or what.

I've added the public accessibility metadata project email list and public-vocabs list to this thread.

I finally figured out what made me uneasy about the proposal - until now I just felt like the dodgy examples I had seen were a symptom rather than just one-off mistakes. I've described the issues below as follows:
- AccessMode (and as a sub-issue, levels of detail). I think this is a serious thing that should be fixed more or less immediately
- AccessFeature. This is important enough to look at immediately and figure out what it really does.
- adaptations. We should let others define this. We really only need "anotherVersion" pointers in most cases
- ATCompatible. I think this is broken, but not critically important. We should fix it when we've done the above stuff
- evolution. We need to have some idea about how to handle this, but a formal specification isn't required IMHO.

= accessMode =

It should be possible for a "single resource" to be available with more than one *set* of accessModes.

I agree and this is the design.  A single resource can require one or more accessMode(s).

Per IMS documentation at the links below, the accessMode property describes "Human sensory perceptual system or cognitive faculty through which a person may process or perceive information."  The links below also provide example use cases:


We have also published a best practices and an implementation guide on the use of accessMode at:
https://wiki.benetech.org/display/a11ymetadata/Practical+Properties+Guide .

All of our resources for assistance and guidance can be found at http://www.a11ymetadata.org/resources/ .

accessMode is derived from the Access For All (AfA) specification by IMS Global and the associated ISO 24751 (Individualized adaptability and accessibility in e-learning, education and training) Part 3 (digital resource description) specification. WGBH’s NCAM and the Inclusive Design Research Center (IDRC) who were both involved in ISO 24751 and AfA were also involved in the crafting of this Schema.org<http://Schema.org> accessibility metadata proposal.  IMS who is the copyright holder on AfA released this proposal under a CC-BY-SA license per Schema.org<http://Schema.org> terms.

We chose to leverage this prior work because Schema.org<http://Schema.org> is meant to address search use cases, which have strong overlap with the AfA use cases related to matching the appropriate resource to a learner based on on their needs and preferences.  AfA/ISO 24751 work is being synchronized with and leveraged by other current projects, such as Indie UI and GPII.  Rich Schwerdtfeger can probably talk to this much more than I can.

Essentially, it is unclear what accessMode really means. At least for the search use case where a user needs to find a suitable resource, I think the only way this can really work is if an accessMode is a set of one or more *required user capabilities*.

Yes, I think that is what we are saying too.  In fact, it seems like what others have understood on the public vocabs list as evidenced by:

To pick some examples:

Alice In Wonderland is available as a resource which only *requires* the ability to read text.

We happen to have a few versions of Alice in Wonderland in Bookshare that have been indexed by Google's Custom Search Engine, so let's first search for a pure text version of it with no visual components:

If you inspect one of those titles using Yandex's webmaster tools or Google's rich snippets tool that lets me just plug in a URL parameter, we can see the accessibility microdata on that page:

As you can see we offer 4 different adaptations of that title, which we originally adapted from a physical book that we scanned.  For each adaptation we describe the accessModes that are used.

If I wanted to search for Alice in Wonderland as an audio book in Bookshare we can do that and find that same title, since we offer audio only adaptations on that page, such as DAISY Audio and MP3:

Here’s another example that searches for books about history in Bookshare where the images are either described with short (i.e., alt) or long descriptions (i.e., longdesc or DAISY prodnote):

This applies to several versions available from Project Gutenberg - some of them have images, some don't, but in any case it is possible to understand Alice in Wonderland without the pictures.

This is arguably not the case with kipling's "Just So Stories" (likewise freely available), since there are intrinsic references to the images - although it might be the case that the references are sufficiently descriptive

If there is a movie of Alice In Wonderland available with captions, then there are two sets of requirements. One is to be able to watch a movie and listen to the audio, and the second set is to be able to watch a movie and read the captions. (If it had descriptive audio, a third would be being able to hear the descriptive audio and the main audio…)

That is the purpose of mediaFeature.  A movie typically has accessModes of visual and auditory.  If it has captions, we simply add a mediaFeature of captions and/or a mediaFeature of audioDescription.  You can find an example in our Practical Properties Guide:


Khan Academy has been collaborating with us on implementing this for their videos and Gooru Learning’s search engine will be indexing this metadata to offer their users the ability to filter search results to only videos that are captioned.  We hope that Google will fix a “bug” with their current video search, which is demonstrated by the below search on Wikimedia Commons for captioned videos of the moon landing:


Though I checked and found moon landing videos on the Commons site that are captioned, Google only displays results for videos that are captioned from their own YouTube property for which they have that information.

It might be that this assumption is already made in the proposal, but it is not explicit, and the proposal doesn't explain how to deal with a single version that can be used with different sets of capabilities (whereasit includes a way to point to other versions)

Schema.org<http://Schema.org> allows for multiple values for the same property.  For example, take a look at the example properties for ingredients at the bottom of this page: http://schema.org/Recipe

== details in accessMode ==

the top-level accessMode is only useful for "most" cases. Knowing that I am physically capable of watching and listening to a video is great, unless it is only available in a format I cannot use such as "flash for my iPhone 3", "ogg theora/vorbis"….

Schema.org<http://Schema.org> already has a properties for formats (e.g., http://schema.org/encodingFormat , http://schema.org/BookFormatType )

Similarly if I have more detailed requirements. For example although I can watch and hear a video tutorial, being red-green colourblind or requiring very high contrast may mean I can't actually understand it in practice.

For those use cases, we have accessMode = colorDependent and mediaFeature =

= accessFeature =

It isn't clear what the requirements and use cases are for accessFeature.

Are these just cataloguing information, similar to noting that a physical book has 5 pages of color photographs, or a guarantee that e.g. each image has alternative text, or a longer description is provided as necessary?

We thought that implementing AfA or ISO 24751 verbatim for the purposes of web search and for use by web masters would be too complicated: it needed to be a small set of properties that a user could deal with. We decided to combine various ideas into mediaFeature.  mediaFeature simply states that the author/publisher or someone who modified that book, tried to take advantage the stated accessibility features.  For example we make no promise that every image was described appropriately, since it is hard to define that by a single person and actually different consumers of the information may have different opinions about the quality and coverage of the image descriptions.

It seems that they are the former - interesting information that might lead to expectations of the resource described. They don't seem well-enough specified to know if I can rely on them. And the list seems somewhat arbitrary.

It isn't clear, for example, how to classify a resource that assumes the ability to touch several points of a screen at once, which is now a common feature on devices but poses an obvious potential problem to some users such as those who don't have very many fingers.

In this use case, the resource that needs to be described is an application, not the content.  If the accessibility API of this application supports multi-touch with the user’s AT, then the proposed ATCompatible and accessAPI properties should be relevant.  Furthermore, we also have a property called controlFlexibility, which also should provide information about whether the application can be fully controlled either via audio, mouse, keyboard, touch or video/gestures.

= adaptations, other versions, ... =

Liddy has pointed out that the concept of "the original" is only sometimes interesting, and that it isn't an accessibility concept but a more general one related to creative works.

Actually to a teacher, the “original” is important if they are trying to find a resource for a disabled student that is an adaptation of the “original” resource used by the rest of the students in the classroom who don’t have a disability.  Similarly a publisher or book seller wants to make it easy for consumers with specialized needs to find resources that have been adapted or “made accessible.”  These adaptations properties are also derived from AfA and ISO 24751 and I think it’s worthwhile reading their specifications to understand  their reasoning. We did simplify the terms (removing partial adaptation and the like) to simplify this for search purposes.

I think it sometimes matters and sometimes doesn't ("born-digital" works don't necessarily have a canonical original form, but may be served in very different forms to different users).

I think we should generally not try to coin accessibility-specific ways of talking about different versions of acreative work. FRBR and its many analogous efforts already deal with this issue, and we should just use whatever schema.org<http://schema.org> adopts where we need it. A generic "anotherVersion" relation is probably the most commonly required thing anyway.

This isn't high priority, except that I think we should quickly decide we want to avoid dealing with defining stuff that others have spent decades on, and instead try to use their work...

= ATCompatible =

being a boolean seems a bit naive. I can think of lots of resources that work with specific assistive technologies, but fail with others (my instant example is http://cookiecrook.com/longdesc/iframe/ which works great on VoiceOver but as far as I can tell does nothing to work effectively with the built-in MacOS magnifier - two assistive technologies I have readily available…).

In the case of content, we once again, we tried to simplify the work for web masters and searchers.  Since we can always expand the vocabulary, we decided to err on simplicity, and to just have the focus on whether accessible technology was considered; enumerating WCAG sections would be overly daunting in a search. In the case of applications, we did provide greater granularity with the accessAPI property, which would differentiate what accessibility API, such as VoiceOver, the application is compatible with.

I don't think ATCompatible is the first priority for fixing. But I do think it should be replaced with something because I don't think it is very useful.

= Evolution =

Who looks after the terms, and how do they get extended?

This is probably the problem with any Schema.org<http://Schema.org> proposal.  By basing this proposal on existing standards, such as AfA and ISO 24751, it will be easier to evolve these terms as more people and organizations (e.g., IMS, GPII, IDPF) will be invested in keeping these terms synchronized with other standards and technologies. We believe that there are some efforts with 24751 to have a registry to assist in this process.

Some things are pretty basic and could be reliably laid down - the ability to see or hear is something that we more or less understand. But others, such as interacting with new types of device, will probably change. Pointer and especially touch interfaces have changed a lot in this century, voice interfaces are starting to do so, and gestural things like Wii and Kinect are beginning to appear more often too.

That's enough comment for now. Hopefully I've clarified the main threads that I think need action (some more urgently than others).



Charles McCathie Nevile - Consultant (web standards) CTO Office, Yandex
     chaals@yandex-team.ru<mailto:chaals@yandex-team.ru>         Find more at http://yandex.com
Received on Thursday, 5 September 2013 17:52:59 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 21:29:30 UTC