Re: Thoughts on the future of EPUB 3

Sorry to interrupt and for being that guy throwing technical details
into this thread, but I'd like to add some shades of gray to the
800-pound gorilla and its new format.  

Please be assured this isn't about begging to differ but I guess it is
of interest as it is related to authoring freedom + Reading Systems
constraints.  

I, amongst others, actually retro-engineered their new format. All
thanks and acknowledgments should go to J. Howell though, who put an
awful lot of efforts to document it there
(https://www.mobileread.com/forums/showthread.php?t=263902) and created
plugins displaying errors the Gorilla's official tools won't even
display!  

Dave (Cramer) can tell a lot of stories about this new format, I know
for instance that he had to fight in order for <caption> (as in table
caption) to be supported. That's because their KFX format is not
HTML/CSS/JS. At all.  

Their new format is a binary representation of JSON, which is built on
top of Amazon Ion (https://amzn.github.io/ion-docs/index.html). 

What it means is that they deal with EPUB contents beforehand, rendering
files in a headless browser (Phantom.js, as far as I can tell), checking
tags and styles, and turning them into JSON fragments. 

They then inject those JSON fragments that they entirely control during
runtime, and can do whatever other Reading apps can't necessarily do
because they can't preprocess files (e.g. removing drop caps or floats
when the user reaches a font-size ceiling, improving contrast, etc.).  

To put it simply, it means they support a subset of HTML5 and CSS3, and
have to manually implement it in their converter (cf. Dave's <caption>).


It also means EPUB files are never rendered to the user as intended,
because the Gorilla have the luxury to filter a lot of stuff
(inline-block, some media queries, scripts, tags, etc.) and edge cases
will make the conversion to the new format fail. 

As a matter of fact, if authors do funky stuff which doesn't fit in the
use cases they implemented, it is _de facto_ an edge case - some people
on eprdctn can testify.  

I can also mention their support for RTL scripts is appalling because of
this technical choice, with users vocally and publicly complaining about
that. For the record, it took them 10 years to finally support it
(limited to Word files as a source). 

On the other hand, they can do stuff like full-bleed, figures, wrap,
regions, etc. cf.
https://medium.com/@jiminypan/kindle-in-motion-cbb1c3f7c4e5. But please
bear in mind it comes at a huge price in terms of "authoring freedom",
agile implementation of every new spec, etc. 

They don't support flexbox, nor Grid, nor OpenType Features, nor
transparent PNG, and so on and so forth. Their new format is not aligned
with the web, it is evolving in parallel, in a completely different
timeline, with developments the web platform will never benefit from,
imposing a burden on authors no other Reading System can impose. 

So it's all about balance. Either you want to be able to do anything
(and CSS is an absolute beast to manage for example, even simple stuff
like managing the font-size the user sets feels like going through Hell
because of bad practices and the lack of standard APIs) or Reading
Systems have to set constraints.  

FYI, in the context of Readium CSS, I've been pushing for research
related to interop/compatibility. I am convinced this is the root of a
lot of issues but quite frankly, the KFX (Kindle's new format) diverge
from standards so much that it's not even worth documenting it, because
this is a completely different model other Reading Systems/Apps just
can't enforce. You can take a look at the features for sure, but their
reality is so different that it can't necessarily translate into others.


Best,  

Jiminy 

Le 19.01.2018 21:01, McCloy-Kelley, Liisa a écrit :

> David- 
> 
> I'm going to respectfully disagree with you. I have a very different opinion of what is the 800-pound gorilla issue that we face at this point. 
> 
> The "huge player" to whom you refer no longer works the way they once did and is certainly not holding the market back with any kind of inferior reading experience. In fact, they not only accept EPUB for their ingestion, they depend on it and are pushing for content providers to get better at it. They use the epub markup for the footnote popups and have implemented this as well (or better) than any other reading system. They use the page anchors we include to improve navigation. In fact, Penguin Random House sends them only EPUB 3 for all of our reflow titles and even for most of our fixed page children's picturebooks with javascript pop-ups for text enlargements. And they work! 
> 
> Ok, so maybe they don't directly sell EPUB. But they also aren't sending old mobi files to any new device any more. I don't know how much you've been looking at their reading system over the past year, but while everyone else was focused elsewhere, they implemented a new and really functional reading system across their entire device landscape, a brand new proprietary format and they swapped what seems like the ENTIRE back catalog to update it. I've watched as my library of years' worth of ebooks has had files update to look and work better. 
> 
> You should take a look. They have a really good reading experience and the content displays better than most (not always, but often). 
> 
> Everyone here needs to stop worrying about whether they are or were being held back by said "huge player" and start figuring out how to catch up and try to get ahead-- with the content we make and with the reading platforms we develop.   
> 
> LIISA MCCLOY-KELLEY 
> 
> VP, Director Ebook Product Development & Innovation, PRH 
> 
> lmccloy-kelley@penguinrandomhouse.com    
> 
> FROM: David Herron <david@davidherron.com>
> DATE: Friday, January 19, 2018 at 1:03 PM
> TO: Dave Cramer <dauwhe@gmail.com>
> CC: "public-epub3@w3.org" <public-epub3@w3.org>, "public-publishingbg@w3.org" <public-publishingbg@w3.org>
> SUBJECT: Re: Thoughts on the future of EPUB 3
> RESENT-FROM: <public-publishingbg@w3.org>
> RESENT-DATE: Friday, January 19, 2018 at 1:27 PM 
> 
> That's an interesting read - as the others have said.  There is a value in maintaining backward compatibility so long as that doesn't become a heavy albatross around your neck dragging you under.. 
> 
> To me the post misses the big 800 pound gorilla issue - namely that a huge player in the eBook market is not using EPUB at all, but sticking with a proprietarized version of MOBI, and their very popular eBook reader does not support EPUB.  That means the eBook market is being held back by inadequacies of the technology supported by that huge player.   
> 
> By adopting HTML5 and modern JavaScript and CSS techniques, EPUB has the potential to deliver a truly remarkable reading experience with interactive books and more.  Those Harry-Potter-like newspapers with moving video and whatnot could be a reality and not limited to magical people. 
> 
> But I suspect the eBook market is overly focused on the limited capabilities of the eBook platform mentioned earlier. 
> 
> I'd seen a blog post by someone a few months ago claiming that eBook Reader Innovation has "stalled".  I wrote two blog posts in response along the lines of what I just said.  The market hasn't stalled, it's being held back. 
> 
> * https://techsparx.com/blog/2017/10/kindle-epub.html
> * https://techsparx.com/blog/2017/10/e-reader-innovation.html
> 
> FWIW I write this as the implementor of a toolchain that can take a bundle of Markdown/CSS/etc files as input and produce either an EPUB3 or website representation of an eBook.  https://akashacms.com/ [1] if you're interested.  An example eBook I've published that way is at https://greentransportation.info/ev-charging/toc.html [2] 
> 
> + David Herron 
> 
> On Thu, Jan 18, 2018 at 7:36 AM, Dave Cramer <dauwhe@gmail.com> wrote:
> 
>> Inspired by the recent debate about EPUB 3.1 and backward compatibility, I wrote a blog post on the future of EPUB 3, compatibility, mistakes, and even old versions of EPUB being "good enough." I think there is much we can learn from the web on this subject.  
>> 
>> http://epubsecrets.com/good-enough-a-meditation-on-the-past-present-and-future-of-epub.php 
>> 
>> Thanks, 
>> 
>> Dave

 

Links:
------
[1] https://akashacms.com/
[2] https://greentransportation.info/ev-charging/toc.html