Accessing Graphics

Hi;

Diagrams are a prevalent form of communication in contemporary society.  They are used to explain task sequences, convey concepts, even design interactions between classes in a UML sequence diagram, yet they remain one of the last frontiers in the world of accessible information.  This need not necessarily be the case, as after all, diagrams are just a transport mechanism for meaning as much as the words on this page are.

There's three main ways in which I consider diagrams can be made accessible.  Each involves extracting the meaning from it's diagramatical encoding, but each differ in where that decoding takes place.

A bit of communications theory.
There's various communications models used to explain how people communicate with one another and technical communications systems.  One of these, which is classed as a transmission model of communication, is Claude Shannon's 1948 model that featured in the Bell Systems Journal of that year.  Shannon proposed that there were five stages to communication:
1. A sender considers the meaning to be sent
2. That meaning is encoded into some physical form
3. The physical representation of the meaning is transmitted to a receiver, using physical communication channels
4. The physical representation is decoded to expose the transmitted meaning
5. The receiver then absorbs the transferred meaning

If we apply this to diagrams, the lines, colors, spatial relationships are purely encoding, and are distinct and separate from the meaning they convey.  Therefore, to get at the meaning, all that needs to be done is to decode the physical representation of that meaning.

It's this decoding that causes problems in accessibility.  Psychology examines the process of receiving meaning in a bit more detail.  According to psychology, we first receive sensory stimuli, which can be in the form of waves, particles or contact with other physical objects.  We then automatically group these into perceptual groups, which in the world of diagrams would be the lines, shapes, colors, words, etc. that form a diagram.  The final stage in this process is for us to cognitively associate meaning with those perceptual groups.

Examining the psychological process of receiving information, there's two main problematic areas for accessibility.  Either people can't receive the sensory stimuli due to physical, environmental or other constraints, or they cannot cognitively associate meaning with the perceptual groupings, which may be due to one of a number of factors.

Semantics can resolve both of these issues.  If it's embedded as part of the physical transportation medium, then the transferred meaning can be reassembled in any form suitable for the user.  This could be a form that bypasses problems, be they physical, environmental or whatever in nature, that prevent the user from receiving sensory stimuli, or it could be a form adapted to allow the user to cognitively associate meaning with the perceptual groups, where they may have been unable to with the original perceptual groupings.  Most people are familiar with the fact that some people cannot receive certain types of sensory stimuli, the blind cannot receive light waves, the deaf sound waves, and so on, or it may be in appropriate for people to receive certain types of stimuli, well, you need to look where you're going when walking, you may fall down some steps.  However, accessibility goes further than just dealing with issues of disability, be it permenant or physical, the ultimate aim of accessibility is to ensure everyone can access meaning.  This includes adapting the encoding of the physical representation, but not the type of stimuli used to encode it.  For example, a blue line would yield no meaning to someone unfamiliar with the UK's Ordenance Survey 1:50000 maps, yet it represents a motorway.  This is because they haven't learnt the particular set of symbolic encodings used in an OS 1:50000 sheet.  Through the use of semantic content adaptation barriers such as this lack of knowledge of various symbolic encoding sets can be overcome.

Semantic content need not necessarily be encoded in the physically transported content, it can be gained after transportation.  The final stage of the psychology sequence involves associating cognitive meaning with sensory stimuli, or in other words, extracting the semantics from the content.  This process can be automated by intelligent agent software that have been taught the encoding techniques used in a particular diagramatical context, and this set of extracted semantics can then be reencoded as if the semantics were originally embedded within the transported physical representation.

Finally, and to me the most fun, as I've been working on this in industry, is adaptation of the sensory stimuli itself.  This is only suitable for those unable to receive the sensory stimuli for whatever reason, and involves converting diagrams and images into a form of sensory stimuli that the intended receiver can receive.  However, as this is intended more for a protocols and formats audience and not an ATIA one, I'll leave the detail out for this.

Will