Re: [a11y-metadata-project] Re: Schema.org accessibility proposal Review...

I just realized we have a major difference of interpretation regarding accessMode, perhaps created by prior experience with AfA.  To use the example below:

<div itemscope=”” itemtype=”http://schema.org/Movie”>
<meta itemprop=”accessMode” content=”visual”/>
<meta itemprop=”accessMode” content=”auditory”/>
<meta itemprop=”mediaFeature” content=”audioDescription”/>
<meta itemprop=”mediaFeature” content=”captions”/>
</div>

The Schema.org proposal intends for this to be interpreted as:
- it's a movie
- the movie has visual and auditory content.  the visual content is complemented by the auditory content and neither the visual or auditory content can stand alone.
- the movie has captions for the audio content
- the movie has audioDescription for the visual content

A silent movie, would not have accessMode auditory and therefore likely would not require captions for it to be accessible.

accessMode has nothing to do with how the resource has been adapted.  It is simply a statement of the combination of "human sensory perceptual systems or cognitive faculties" required to access the original/unadapted resource.

mediaFeature(s) cover the accessibility features that have been implemented in the adapted or born accessible resource.

More comments below to Chaals.  I didn't respond to every comment, because they may not be relevant given the major difference of interpretation I just realized we have.

On Sep 7, 2013, at 5:49 AM, Andy Heath <andyheath@axelrod.plus.com>
 wrote:

> 
> <Lots of snipping>
> 
> Chaals quoted (and wrote a little bit of):
> 
>>>> = accessMode =
>>> 
>>>> It should be possible for a "single resource" to be available with
>>>> more than one *set* of accessModes.
>>> 
>>> I agree and this is the design.  A single resource can require one or
>>> more accessMode(s).
>> 
>> Yes, but...
>> 
>>> … the accessMode property describes "Human sensory perceptual system
>>> or cognitive faculty through which a person may process or perceive
>>> information." […]
>>> We have also published a best practices and an implementation guide on
>>> the use of accessMode at:
>>> <http://www.a11ymetadata.org/wp-content/uploads/2013/04/A11yMetadataProjectBestPracticesGuide_V.6.pdf>
>>> 
>>> <https://wiki.benetech.org/display/a11ymetadata/Practical+Properties+Guide>
>>> 
> 
> Chaals wrote:
> 
>> Yep. But that has an example which I'll use:
>> 
>> A movie with captions and extended audio description would be encoded as
>> follows
>> <div itemscope=”” itemtype=”http://schema.org/Movie”>
>> <meta itemprop=”accessMode” content=”visual”/>
>> <meta itemprop=”accessMode” content=”auditory”/>
>> <meta itemprop=”mediaFeature” content=”audioDescription”/>
>> <meta itemprop=”mediaFeature” content=”captions”/>
>> </div>
>> 
>> My first impression is that if the video has good audio description,
>> then claiming it has accessMode "visual" seems wrong, since you don't
>> need to see it. Likewise, since it is captioned, it seems you don't need
>> to hear it.
>> 
>> So it doesn't have a single *required* accessMode. On the other hand,
>> you need to *either* see (clearly enough) or hear, in order to get the
>> content.
> 
> The interpretation we had in AfA 3.0 of each property like this was that 
> each specified not "accessMode required" but instead "accessMode 
> available".  Did this project take a different interpretation ?
> 
> I haven't yet read the rest of this because I'm trying to focus on the 
> same ISO meeting Chaals is in (but will do so) - this interpretation is 
> so crucial as to change the whole emphasis so I wanted to reply quickly 
> on this one point.
> 
> andy
> axelrod access for all
> DiversityNet
> http://axelafa.com
>>> accessMode is derived from the Access For All (AfA) specification...
>> 
>>> We chose to leverage this prior work because
>>> Schema.org<http://Schema.org> is meant to address search use cases,
>> 
>> Yes, and I think that was the right choice. (Right now I am in an ISO
>> meeting on this very topic).
>> 
>>>> Essentially, it is unclear what accessMode really means. At least for
>>>> the search use case where a user needs to find a suitable resource, I
>>>> think the only way this can really work is if an accessMode is a set
>>>> of one or more *required user capabilities*.
>>> 
>>> Yes, I think that is what we are saying too.  In fact, it seems like
>>> what others have understood on the public vocabs list as evidenced by:
>>> http://lists.w3.org/Archives/Public/public-vocabs/2013May/0092.html
>> 
>> Not quite. The example there suggests (based on the documentation of
>> what accessMode means) that you need both visual and tactile
>> interactivity to work with the resource.
>> 
>> But I agree that what people understand in the examples quoted is that
>> there are multiple ways to access some things.
>> 
>>> http://lists.w3.org/Archives/Public/public-schemabibex/2013Aug/0049.html
>> 
>> I'll leave Karen to answer on that one…
>> 
>>>> To pick some examples:
>>> 
>>>> Alice In Wonderland is available as a resource which only *requires*
>>>> the ability to read text.
>>> 
>>> We happen to have a few versions of Alice in Wonderland in Bookshare
>> 
>> Yep.
>> 
>>> If you inspect one of those titles using Yandex's webmaster tools or
>>> Google's rich snippets tool that lets me just plug in a URL parameter,
>>> we can see the accessibility microdata on that page:
>>> http://www.google.com/webmasters/tools/richsnippets?q=https%3A%2F%2Fwww.bookshare.org%2Fbrowse%2Fbook%2F18041<http://www.google.com/webmasters/tools/richsnippets?q=https://www.bookshare.org/browse/book/18041>
>>> 
>> 
>> I looked at https://www.bookshare.org/browse/book/268061 with
>> http://webmaster.yandex.ru/microtest.xml
>> 
>>> As you can see we offer 4 different adaptations of that title, which
>>> we originally adapted from a physical book that we scanned.  For each
>>> adaptation we describe the accessModes that are used.
>> 
>> Yes. But
>> <https://www.bookshare.org/download/book?titleInstanceId=268061&downloadFormat=DAISY_WITH_IMAGES>
>> doesn't actually *require* visual capability, does it?

Actually, it does require visual capability if none of the images are described.  We don't have either mediaFeature equal to alternativeText and/or longDescription.  Unfortunately, Bookshare doesn't yet have the capability to automatically scan the text to look whether the alt(s) for the images have sufficient descriptions.  We do know whether a volunteer went through and added DAISY prodnotes (somewhat similar to aria-describedby) to describe the images.

>> 
>> [...]
>>>> If there is a movie of Alice In Wonderland available with captions,
>>>> then there are two sets of requirements. One is to be able to watch a
>>>> movie and listen to the audio, and the second set is to be able to
>>>> watch a movie and read the captions. (If it had descriptive audio, a
>>>> third would be being able to hear the descriptive audio and the main
>>>> audio…)
>> 
>> (See also the movie example mentioned earlier)
>> 
>>> That is the purpose of mediaFeature.  A movie typically has
>>> accessModes of visual and auditory.  If it has captions, we simply add
>>> a mediaFeature of captions and/or a mediaFeature of audioDescription.
>>> You can find an example in our Practical Properties Guide:
>>> 
>>> https://wiki.benetech.org/display/a11ymetadata/Practical+Properties+Guide#PracticalPropertiesGuide-Video
>>> 
>> 
>> Yes, but I don't think that is actually correct. If the movie has
>> captions, you *don't* necessarily need auditory capacity anymore.
>> However, in order to watch the movie without auditory capability, you
>> may need the ability to read images of text (this also depends on the
>> format used for captioning...)
>> 
>>>> It might be that this assumption is already made in the proposal,
>> 
>> It certainly seems to be an assumption.
>> 
>>>> but it is not explicit, and the proposal doesn't explain how to deal
>>>> with a single version that can be used with different sets of
>>>> capabilities (whereas it includes a way to point to other versions)
>> 
>>> Schema.org<http://Schema.org> allows for multiple values for the same
>>> property.  For example, take a look at the example properties for
>>> ingredients at the bottom of this page: http://schema.org/Recipe
>> 
>> Sure. But while the data about Alice in Wonderland shows how to include
>> multiple alternative books, each with a *set* or properties, I am not
>> sure the accessMode definition works for including different
>> combinations that are "mutually exclusive".
>> 
>> To continue the captioned, audio described movie example, I think we
>> should be saying it has an accessMode which requires the ability to see
>> and hear, a different accessMode that requires the ability to hear, and
>> a third one that requires the ability to see and to read text on images.
>> 
>> Any *one* of these is sufficient for a user to successfully use the
>> video resource, no?

See comments at the top of the email regarding the interpretation of accessMode.

>> 
>>>> == details in accessMode ==
>>> 
>>>> the top-level accessMode is only useful for "most" cases. Knowing
>>>> that I am physically capable of watching and listening to a video is
>>>> great, unless it is only available in a format I cannot use such as
>>>> "flash for my iPhone 3", "ogg theora/vorbis"….
>>> 
>>> Schema.org<http://Schema.org> already has a properties for formats
>>> (e.g., http://schema.org/encodingFormat ,
>>> http://schema.org/BookFormatType )
>> 
>> Yep. So we should be expecting those to be used, right?

We do, however for books/textbooks the thinking is evolving and I'll let Matt Garrish comment on that.

>> 
>>>> Similarly if I have more detailed requirements. For example although
>>>> I can watch and hear a video tutorial, being red-green colourblind or
>>>> requiring very high contrast may mean I can't actually understand it
>>>> in practice.
>>> 
>>> For those use cases, we have accessMode = colorDependent
>> 
>> Yes. But this assumes that users who require white-on-black, users who
>> use a common high-contrast mode, and users who are unable to distinguish
>> light blue from white have the same requirements.
>> 
>>> and mediaFeature =

Should have been mediaFeature=highContrast and it may also require mediaFeature=audioDescription

>> 
>> Something missing here.
>> 
>>> = accessFeature =
>> 
>>>> Are these just cataloguing information, similar to noting that a
>>>> physical book has 5 pages of color photographs, or a guarantee that
>>>> e.g. each image has alternative text, or a longer description is
>>>> provided as necessary?
>>> 
>>> We thought that implementing AfA or ISO 24751 verbatim for the
>>> purposes of web search and for use by web masters would be too
>>> complicated: it needed to be a small set of properties that a user
>>> could deal with.
>> 
>> Hmmm. It needs to be as simple as possible - and no simpler...
>> 
>>> We decided to combine various ideas into mediaFeature.
>> 
>> Which I think makes sense.
>> 
>>> mediaFeature simply states that the author/publisher or someone who
>>> modified that book, tried to take advantage the stated accessibility
>>> features.
>> 
>> Right.
>> 
>>> For example we make no promise that every image was described
>>> appropriately,
>> 
>> But is best practice to use mediaFeature: textAlternative when there is
>> at least one image with an alternative, or when all the images have one?

mediaFeature textAlternative and/or longDescription means the publisher tried to make all the images accessible.  There may have been images the publisher determined did not require alternative descriptions.

>> 
>> (We can't guarantee that everyone does everything perfectly, but it is
>> important to explain what we think people should be trying to do as well
>> as they can).
>> 
>>>> It isn't clear, for example, how to classify a resource that assumes
>>>> the ability to touch several points of a screen at once, which is now
>>>> a common feature on devices but poses an obvious potential problem to
>>>> some users such as those who don't have very many fingers.
>>> 
>>> In this use case, the resource that needs to be described is an
>>> application, not the content.
>> 
>> Hmm. I don't understand the difference between applications and content.
>> I can see there are some things I would call one, and some that I would
>> call the other, but resources identified by schema.org seem hard to
>> classify into one or the other.
>> 
>> For example, is an interactive map an application or content?

This will get harder and I think interactive is going to make a lot more content appear as applications.  However, there are clear cases where you want to focus on applications, such as describing whether the apps in an app store are taking advantage of the accessibility APIs the platform offers.  This would be fantastic to have in both the Apple App Store and for apps in Google Play.

>> What about
>> a document that allows structured navigation, a la DAISY?

This is purely content.

>> 
>> I don't know of a clear way to determine the difference. But then, I am
>> not convinced that we need one.
>> 
>>> If the accessibility API of this application supports multi-touch with
>>> the user’s AT, then the proposed ATCompatible and accessAPI properties
>>> should be relevant. Furthermore, we also have a property called
>>> controlFlexibility, which also should provide information about whether
>>> the application can be fully controlled either via audio, mouse,
>>> keyboard, touch or video/gestures.
>> 
>> Yes.
>> 
>>>> = adaptations, other versions, ... =
>>>> 
>>>> Liddy has pointed out that the concept of "the original" is only
>>>> sometimes interesting, and that it isn't an accessibility concept but
>>>> a more general one related to creative works.
>>> 
>>> Actually to a teacher, the “original” is important if they are trying
>>> to find a resource for a disabled student that is an adaptation of the
>>> “original” resource used by the rest of the students in the classroom
>>> who don’t have a disability.
>> 
>> I don't think so, I believe they are happy to know that two things are
>> versions of each other without caring which was the original.
>> 
>>> Similarly a publisher or book seller wants to make it easy for consumers
>>> with specialized needs to find resources that have been adapted or “made
>>> accessible.”
>> 
>> But again, the publisher doesn't really care which is the "original" -
>> and in the case of modern electronic works, the concept may not have a
>> real meaning.
>> 
>>> These adaptations properties are also derived from AfA and
>>> ISO 24751 and I think it’s worthwhile reading their specifications to
>>> understand their reasoning. We did simplify the terms (removing partial
>>> adaptation and the like) to simplify this for search purposes.
>> 
>> There is a lot of literature, and there are a number of standards,
>> describing how instances of a CreativeWork relate to each other. I don't
>> think we *need*, for accessibility, anything more than the super-simple
>> "this is a version of that". If people need to describe the relationship
>> for other reasons, they should use something that can support the things
>> they may need - i.e. the stuff that got removed (and probably stuff that
>> was never in AfA but is in Bibex, or FRBR…)
>> 
>>>> This isn't high priority, except that I think we should quickly
>>>> decide we want to avoid dealing with defining stuff that others have
>>>> spent decades on, and instead try to use their work...
>> 
>> This isn't high priority, except that I think we should quickly decide
>> we want to avoid dealing with defining stuff that others have spent
>> decades on, and instead try to use their work...
>> 
>>>> = ATCompatible =
>>> 
>>>> being a boolean seems a bit naive. I can think of lots of resources
>>>> that work with specific assistive technologies, but fail with others
>>>> (my instant example is http://cookiecrook.com/longdesc/iframe/ which
>>>> works great on VoiceOver but as far as I can tell does nothing to
>>>> work effectively with the built-in MacOS magnifier - two assistive
>>>> technologies I have readily available…).
>>> 
>>> In the case of content, we once again, we tried to simplify the work
>>> for web masters and searchers. Since we can always expand the vocabulary,
>> 
>> This is why I think fixing this bit is not the first priority.
>> 
>>> we decided to err on simplicity, and to just have the focus on whether
>>> accessible technology was considered;
>> 
>> Yes, but
>> 
>>> enumerating WCAG sections would be overly daunting in a search.
>> 
>> Well, it would be a fairly unfriendly way to define things.
>> 
>>> In the case of applications, we did provide greater granularity with the
>>> accessAPI property, which would differentiate what accessibility API,
>>> such as VoiceOver, the application is compatible with.
>> 
>> Yes. Because for an actual user, it doesn't matter that the developer
>> considered assistive technology in general, only whether they can use a
>> resource with their particular setup. This is analagous to the reason
>> why knowing the format of a resource is important.
>> 
>>> I don't think ATCompatible is the first priority for fixing. But I do
>>> think it should be replaced with something because I don't think it is
>>> very useful.
>>> 
>>>> = Evolution =
>>> 
>>>> Who looks after the terms, and how do they get extended?
>>> 
>>> This is probably the problem with any Schema.org<http://Schema.org>
>>> proposal.
>> 
>> Sure. Which is why I ask the question in this particular case :)
>> 
>>> By basing this proposal on existing standards, such as AfA and ISO
>>> 24751, it will be easier to evolve these terms as more people and
>>> organizations (e.g., IMS, GPII, IDPF) will be invested in keeping
>>> these terms synchronized with other standards and technologies.
>> 
>> My experience suggests that this will actually make it harderif the
>> different standards aren't already evolving in lock-step, unless we have
>> a clear story for what triggers changes in the schema.org version.
>> 
>>> We believe that there are some efforts with 24751 to have a registry
>>> to assist in this process.
>> 
>> There are efforts in 24751 to have a registry, but I am not sure that
>> its purpose is to facilitate keeping schema.org as useful as possible in
>> the face of the evolving world.
>> 
> 
> 
> 
> 
> andy
> andyheath@axelrod.plus.com
> -- 
> __________________
> Andy Heath
> http://axelafa.com
> 
>