Re: The AAP EPUB 3 Implementation Project

Hi Gerardo -

Thanks for sending all of this information! I wish I had more time to go over it before our webinar tomorrow… Thanks for bringing me up to speed on some of the conversations that happened after EDUPUB regarding MathML. You may have noticed that I was in attendance at EDUPUB, too. In fact, EDUPUB was where I discovered that Sanders had come up with an excellent case study which he will be presenting in abbreviated form at tomorrow’s webinar.

I’m very interested to discuss further and will get back to you as soon as I’ve had a chance to review (which won’t be before the end of this week at earliest). I also know of an additional publisher who may be interested in hearing about the toolchain that accepts MathML and generates SVG/PNG images of the math with descriptions that can be voiced by a screenreader.

In fact, the company that I’m thinking of is already in a position where they have the MathML, and they have the images. They have a plan to get the descriptions to be voiced by a screenreader, but they have not yet done the implementation. Content is currently distributed on a very large personal learning environment portal built by the publisher. Your bridge idea may very well be something that my contacts at this company want to hear about, too.

I’ll check in with you again by the end of this week/beginning of next to discuss my thoughts and potential next steps for introducing you to the company I have in mind.

Thanks again for bringing these discussions and ideas to my attention!

Jean Kaplansky
Solutions Architect
Aptara, Inc.
Email: jean.kaplansky@aptaracorp.com<mailto:jean.kaplansky@aptaracorp.com>
Skype: JeanKaplansky
Mobile: 518 487 9670

[cid:E648636A-E12A-42CE-A275-42D343FE80E0]

From: Gerardo Capiel <gerardoc@benetech.org<mailto:gerardoc@benetech.org>>
Date: Tuesday, November 5, 2013 at 9:41 PM
To: Jean Kaplansky <jean.kaplansky@aptaracorp.com<mailto:jean.kaplansky@aptaracorp.com>>
Cc: Richard Schwerdtfeger <schwer@us.ibm.com<mailto:schwer@us.ibm.com>>, Ivan Herman <ivan@w3.org<mailto:ivan@w3.org>>, Peter Krautzberger <peter.krautzberger@mathjax.org<mailto:peter.krautzberger@mathjax.org>>, W3C Digital Publishing IG <public-digipub-ig@w3.org<mailto:public-digipub-ig@w3.org>>, W3C Public Digital Publishing IG Mailing List <public-digipub-ig-comment@w3.org<mailto:public-digipub-ig-comment@w3.org>>, Anh Bui <anhb@benetech.org<mailto:anhb@benetech.org>>
Subject: Re: The AAP EPUB 3 Implementation Project

Jean,

I've been collaborating with Peter, Sanders, Wikimedia and others around this toolchain idea, which we see as a bridge until all reading systems provide native MathML visual rendering and accessibility.  I've also spoken to Rich and Neil Soiffer about it.

Based on a prototype that was created by Benetech, Volker Sorge and Connexions engineers at an accessibility sprint, DIAGRAM Center (a collaboration between Benetech, NCAM and the U.S. Fund for DAISY) is planning to fund the American Mathematical Society to enhance MathJax with APIs that will improve the ability for MathJax to be a server-side "toolchain" utility.  We are also working with Volker Sorge who was involved with ChromeVox and others at Google on enabling the aural rendering capabilities of ChromeVox to also easily run within the server-side "toolchain".  This means that we could have a toolchain that accepts MathML and generates SVG/PNG images of the math that have descriptions, which can be voiced by a screenreader.  Ultimately, the accessibility features of SVG that Rich cites below will enable navigation of the math by navigating the SVG.

On our trip back from EDUPUB last week, T.V. Raman and I did some brainstorming on how to further improve upon this idea to provide a bridge that enables accessible math on lower-end or older reading systems, which don't support EPUB 3 or don't have accessible and native MathML support.  The approach that T.V and I identified would enable the same EPUB file that provides this "fallback" functionality to also provide a rich MathML enabled experience. This approach also enables crowdsourced improvement/correction of the semantics of the MathML or the automatically generation descriptions.  This is desirable, because as you may know, though MathML may render correctly visually, it may not be coded in a way that provides enough semantics for a screenreader to correctly speak the math expression.

Our approach is described in the email below that I sent to Sanders for his feedback.  I'd love to hear your thoughts as well:
---------
Sanders,

... T.V Raman and I brainstormed further about how to solve the hybrid EPUB 2/3 problem, which is even a problem with EPUB 3 readers that don't yet support MathML.  Our idea is as follows:

1) When we convert MathML to SVG/PNG, we store the source MathML in a cloud database and maintain a unique locator ID either in the local image filename, alt, some DOM accessible property or ideally within the text attribute in the PNG file:

http://en.wikipedia.org/wiki/Portable_Network_Graphics#Ancillary_chunks
http://blog.nihilogic.dk/2008/08/imageinfo-reading-image-metadata-with.html

I would still recommend we insert into the alt the server-side ChromeVox generated description.

Then I would take one of the following approaches:

2a) We add ChromeVox support to Readium and enhance ChromeVox to recognize images of math that contain a resource locator to the original MathML.  We have experimented with ChromeVox on other WebKit based rendering engines and it seems to work, so this should be possible.

Here's a crude example of injecting ChromeVox's standalone JS code into a page with MathML that could be rendered with Safari/WebKit both visually and aurally:
https://dl.dropboxusercontent.com/u/39156804/androidvox.html

2b) I'm not sure what happens in older EPUB2 readers if you include JavaScript, but if the we can do that, then we could add JavaScript that would invoke ChromeVox in a popup windows that pulls the source MathML.

2c) Hyperlink the image to a web app that pulls the original MathML and accessibly renders it with ChromeVox or Safari on IOS 7/Mavericks.

I'm guessing 2a is the best option, since 2a and 2b have potential visual and functionality issues in order eReaders.  What do you think?

I'd like to go back to the ChromeVox team with this idea based on your input.  One other advantage of a cloud based database is we could correct ChromeVox generated descriptions ideally in the source MathML or do an override of the textual description.  We could also do an end-to-end experiment with 2a on an EPUB 2 that you drop in our system and we render with our web reader.

Cheers,

Gerardo

Gerardo Capiel
VP of Engineering
Benetech

On Nov 5, 2013, at 4:13 PM, "Jean Kaplansky" <Jean.Kaplansky@aptaracorp.com<mailto:Jean.Kaplansky@aptaracorp.com>> wrote:

Hi Richard – Welcome to the group!

I just saw your comments in my inbox, and as I’m getting ready to do a mathml webinar tomorrow, you immediately got my attention…

You say that “A big issue will be getting open source MathML readers (voice) and renders – preferably based on SVG as it is a good fit for mobile.”

Do you mean that it would be better to have MathML that is rendered as SVG than rendered natively in a browser because of the ARIA and tab index support?

The DPUB has started to collect some MathML use cases here: http://www.w3.org/dpub/IG/wiki/MathML_UC. We would definitely like to identify and record additional use cases around MathML and Accessibility. Please go ahead and send email to the group regarding any MathML use cases you wish to add. I’ve been the “de facto” keeper of the MathML use cases for the group to date. (yep. I volunteered. And sort of dragged Peter along with me, since he’s the guy who provided the initial write-ups of the 3 use cases we’ve documented to date…)

Here’s the link to register for the webinar we’re doing tomorrow if anyone’s interested:

https://event.on24.com/eventRegistration/EventLobbyServlet?target=registration.jsp&eventid=681450&sessionid=1&key=BF15E182E16C4F1C3A04C885DF1DC212&sourcepage=register

This was originally going to be a 2 person effort between Peter Krautzberger of MathJax and myself, but we recently expanded our speaker list by one to include Sanders Kleinfeld from O’Reilly who will be presenting a case study about how O’Reilly built a toolchain API to deal with the need to put math on different retail reading devices in addition to mobile apps, the web, and in printed books.

Jean Kaplansky
Solutions Architect
Aptara, Inc.
Email: jean.kaplansky@aptaracorp.com<mailto:jean.kaplansky@aptaracorp.com>
Skype: JeanKaplansky
Mobile: 518 487 9670

<ACD331E8-87DF-4649-A68D-1AC6C320CC10[35].png>

From: Richard Schwerdtfeger <schwer@us.ibm.com<mailto:schwer@us.ibm.com>>
Date: Tuesday, November 5, 2013 at 6:57 PM
To: Ivan Herman <ivan@w3.org<mailto:ivan@w3.org>>
Cc: W3C Digital Publishing IG <public-digipub-ig@w3.org<mailto:public-digipub-ig@w3.org>>, W3C Public Digital Publishing IG Mailing List <public-digipub-ig-comment@w3.org<mailto:public-digipub-ig-comment@w3.org>>
Subject: Re: The AAP EPUB 3 Implementation Project
Resent-From: <public-digipub-ig-comment@w3.org<mailto:public-digipub-ig-comment@w3.org>>
Resent-Date: Tuesday, November 5, 2013 at 6:57 PM


Ivan,

This is a great. Going forward we will be able to expand on ePub's accessibility capabilities. For example, SVG 2 now has ARIA and tabindex in the spec. In the future we will be able to roll this support into books. A big issue will be getting open source MathML readers (voice) and renders - preferably based on SVG as it is a good fit for mobile. Access to digital math by the blind and those with attention deficit would benefit from having this capability. In fact we recently did a study for the Department of Education. The ability for students to have Math navigable through highlighting and speech improved math comprehension by 10-15 percent across the board.


Rich


Rich Schwerdtfeger

<graycol.gif>Ivan Herman ---11/05/2013 05:34:09 AM---I would expect many of you (most of you?) may be familiar with this, but gor those of you who are no

From: Ivan Herman <ivan@w3.org<mailto:ivan@w3.org>>
To: W3C Digital Publishing IG <public-digipub-ig@w3.org<mailto:public-digipub-ig@w3.org>>, W3C Public Digital Publishing IG Mailing List <public-digipub-ig-comment@w3.org<mailto:public-digipub-ig-comment@w3.org>>
Date: 11/05/2013 05:34 AM
Subject: The AAP EPUB 3 Implementation Project

________________________________



I would expect many of you (most of you?) may be familiar with this, but gor those of you who are not:

https://nfb.org/images/nfb/documents/html/aapepub3implementation.xhtml

Lots of good guidelines point on accessibility issues, for example

Ivan

----
Ivan Herman
+31 641044153

(Written on my mobile. Excuses for brevity and frequent misspellings...)




<graycol.gif>