RE: Deep concerns about the future of EPUB

> But, IMO, the WG needs a more complete set of tests if we are going to
promulgate new versions of the spec and certify that there are two complying
implementations.

Absolutely agree. The "test suite" has a bit of a schizophrenic existence
and this question of its purpose seems to come up on both sides
(comprehensiveness and feature support). As someone who contributed to the
original tests, I can state our objective was only to show publishers that
content would render and be an improvement on EPUB 2.

I wouldn't feel bound to making it work for any other purpose than it was
originally conceived. I have no doubt there are better approaches for a
comprehensive test suite, and whether it even meets the needs of showing
epub feature support can also be questioned. I don't see it as a bad thing
to retire it entirely if it's run its useful life, either.

Matt

> -----Original Message-----
> From: Ric Wright [mailto:rkwright@geofx.com]
> Sent: December 29, 2017 12:22 PM
> To: Matt Garrish <matt.garrish@gmail.com>; 'Daniel Glazman'
> <daniel.glazman@disruptive-innovations.com>; 'Bill McCoy'
> <bmccoy@w3.org>; public-publishingbg@w3.org
> Cc: 'Jeff Jaffe' <jeff@w3.org>; 'Tim Berners-Lee' <timbl@w3.org>; eb2m-
> mrt@asahi-net.or.jp
> Subject: Re: Deep concerns about the future of EPUB
> 
> Matt,
> 
> I completely agree with this. The problem I see is TS *has* become more of
a
> can-I-use and simple unit-tests, so it is very useful as a set of
regression tests
> (which is how Readium uses it for the most part).  But, IMO, the WG needs
a
> more complete set of tests if we are going to promulgate new versions of
the
> spec and certify that there are two complying implementations.
> 
> So what DO we need? It would be an excellent exercise to outline in some
> detail what DOES need to be tested and how and what the success criteria
> would consist of. Let me (and hopefully others) give that some thought.
> 
> Thanks
> Ric
> 
> 
> On 12/28/17, 1:08 PM, "Matt Garrish" <matt.garrish@gmail.com> wrote:
> 
> >> Again, to be clear, the TS is an awesome batch of work, it just isnıt
> >enough.
> >
> >It never really had aspirations to be the thing you want it to be, too.
> >I don't recall any reading system developers were even heavily involved
> >in putting it together. Its ambition was to test some essential
> >rendering issues that were poorly supported in EPUB 2 or new to EPUB 3
> >to kickstart and improve support in the early days. It then morphed
> >with the BISG support grid, further taking it into the territory of a
> >caniuse-like input.
> >
> >Matt
> >
> >> -----Original Message-----
> >> From: Ric Wright [mailto:rkwright@geofx.com]
> >> Sent: December 28, 2017 10:26 AM
> >> To: Daniel Glazman <daniel.glazman@disruptive-innovations.com>; Bill
> >> McCoy <bmccoy@w3.org>; public-publishingbg@w3.org
> >> Cc: 'Jeff Jaffe' <jeff@w3.org>; 'Tim Berners-Lee' <timbl@w3.org>;
> >> eb2m- mrt@asahi-net.or.jp
> >> Subject: Re: Deep concerns about the future of EPUB
> >>
> >> I have been following this thread with great interest (Daniel, thanks
> >>for  raising this).  There are a lot of good thoughts and arguments on
> >>all
> >fronts.
> >>
> >> That said, I tend to agree with many of the points Daniel has made
here.
> >> My perspective is from that of a Reading System developer/maintainer.
> >> There are two key points I would make:
> >>
> >> Support for all the varieties of EPUB:
> >> A point I have raised a number of times is that adding new or changed
> >>features to EPUB often sounds like a good idea, but as a RS, we *HAVE*
> >>to  support all variants and features from 2.0 to the latest - now and
> >forever. So
> >> incompatible or changed features are hugely problematic. At this
> >>point,  Readium supports virtually all of 3.0.1 but we havenıt even
> >>considered any
> >of
> >> the new features in 3.1 (weıre short-handed anyway, but even if we
> >> werenıt...).  The net-net is that I, as director of Readium, want to
> >> see
> >what
> >> the value-added is of new features (and that they are being used)
> >> before
> >we
> >> invest the time and effort to support them.
> >>
> >> Validation of Existing Implementations:
> >> At this point, Readium supports around 90% of the features in 3.0.1
> >> as measured by the EPUB TestSuite. But that being said, this isnıt
> >> really a
> >very
> >> good metric.  The EPUB TestSuite is an excellent tool as far as it
> >> goes,
> >but it is
> >> significantly flawed in several ways. Here are three of
> >> them:
> >> 1. Almost 40% of the features it tests are really browser-level
> >functionality.
> >> So it is not really testing the Reading System, but the browser
> >>engine the
> >RS is
> >> built on top of (I think RMSDK was the last RS that built its own
> >>engine).
> >This
> >> is important, of course, but a little misleading as to what is being
> >tested and
> >> how thoroughly.
> >> 2. The features are tested in a very atomic way, one isolated feature
> >>at a  time. This is understandable and the right way to start, but
> >>that vast
> >majority
> >> of bugs and issues Readium encounters are more complex interactions,
> >> or odd variants of features.  I cannot offhand recall a single bug we
> >> have encountered lately that was found through, or could be tested
> >> with, the EPUB TestSuite.  To be clear, the TS is very valuable, it
> >> simply doesnıt
> >go
> >> anywhere near far enough.
> >> 3. There are few if any failure cases.  Or stress cases. The TS is so
> >constructed
> >> such that the tests are clean and well-behaved.  If only the content
> >>we  received from all and sundry were equally well-behaved. :-) The TS
> >>should  also be testing the RS to see how they handle errors.  In some
> >>cases, the  EPUB spec does in fact specify how errors or bad and/or
> >>illegal content  should be handled, but the TS doesnıt test these. One
> >>small example:
> >>For
> >> reasons none of us who were there can recall, declarative SVG
> >>animation is  not allowed in EPUB files.  EPUBCheck dutifully
> >>complains about
> >(declarative)
> >> SVG animation, but the TS doesnıt test that such content is ignored.
> >>The
> >> overhead to disallow such content in a RS is quite large, so Readium
> >simply
> >> doesnıt bother and SVG animation works fine even though it should be
> >>rejected.  Again, to be clear, the TS is an awesome batch of work, it
> >>just
> >isnıt
> >> enough.
> >>
> >> My two cents, FWIW.
> >>
> >> Ric
> >>
> >>
> >>
> >> On 12/27/17, 4:23 AM, "Daniel Glazman"
> >> <daniel.glazman@disruptive-innovations.com> wrote:
> >>
> >> >Le 26/12/2017 à 19:02, Bill McCoy a écrit :
> >> >
> >> >> That being said I think you're engaging in a logical fallacy by
> >> >>claiming that a single mistake by the IDPF EPUB WG over a year ago
> >> >>implies that we must now radically rethink the way W3C
> >> >>Publishing@W3C groups are structured in general, as well as how
> >> >>ongoing development of EPUB 3 will be handled specifically.
> >> >
> >> >I respectfully disagree 100%. If the market is already unable to
> >> >digest
> >> >3.1 six years after the release of 3.0, when will it be able to
> >> >digest 4.0, Web Publications, Portable Web Publications and more?
> >> >Given the EPUB 3 situation, any investment now on WP, PWP, EPUB4 is
> >> >too early and too expensive. The EPUB market is just unable to
> >> >absorb a new spec every three years. Our whole model is flawed.
> >> >
> >> >I suggest we take that chartered time to make a 3.x version conform
> >> >to regular W3C exit criteria, with a test suite and real
> >> >implementation reports, and reach W3C REC (instead of "Recommended
> >> >Specification" IDPF
> >> >designation) instead of diving into useless expensive blue-sky dreams.
> >> >
> >> >*THAT* would be *immensely* useful to *everyone*.
> >> >
> >> >Let me repeat myself on one important point: backwards-compatibility
> >> >requirements of EPUB increase as the EPUB market increase. They will
> >> >become mandatory soon. An ebook is an ebook is an ebook. Nobody
> >> >wants to maintain several technical versions of the same ebook for a
> >> >given technology, nobody wants to update (at a cost) ebooks
> >> >published eons ago that should just work, nobody wants to leave
> >> >legacy reading systems unattended because it could kill the
> >> >ecosystem, and we're not even able to get rid of our ugliest design
> >> >choices any more. Hence my
> >> >conclusion: EPUB is a dead end. A technological disruption will
> >> >happen, it's not a question of "if" any more but only a question of
> "when".
> >> >
> >> ></Daniel>
> >> >
> >>
> >>
> >
> >
>

Received on Friday, 29 December 2017 17:46:49 UTC