Re: Deep concerns about the future of EPUB

Hi Daniel,


I am probably not the person you were hoping to hear from first, but I was co-chair of the 3.1 WG, and I co-chair the Publishing WG, so I have some strong opinions.


I think you raise some excellent points. One thing to keep in mind as we consider the way the EPUB and Publishing ecosystems work is that most books and publications are saleable (at least for now). This adds a layer of complexity to the ecosystem. It is not insurmountable, but it exists.


I will attempt to respond to your questions:


  - why is 3.1 such a failure?
We have not yet released a version of epubcheck that supports 3.1. We are short on developers and/or funding. If we do not get one or the other, it will never gain any traction.

  - why didn't the understandable warnings above serve as showstoppers?
I am partially to blame for this, but at the time, Murata-san's concerns about 3.1 seemed to be more about it being a lot of work than it being a show stopper. The EPUB WG voted on the issue and agreed to move forward. Based on his advice we scaled back the effort considerably. See https://docs.google.com/document/d/152AO1a4FzVZwZs2wmJvbVxK5w9cXAJpCvk5NQOHAlFc/edit?usp=sharing. It was not until discussions of ISO standardization for EPUB 3.1 and EPUB Accessibility arose that this arose to the status of showstopper (within the last few months).

  - conclusions on IDPF standardization Process?
  - what is the next level of EPUB and when is it relevant to
    publish it, industry-wise? EPUB 3.0 was released in October 2011
    and more than six years later, the Industry is NOT ready to move
    to the next version, 3.1, that is almost not implemented at all;
    so does it make any sense to start work on another version?
We cannot affect industry-wide adoption any more than the W3C can make sure that all websites are WCAG AA compliant. Some of the goals of 3.1 were to enable simplified processes for journals. I am using a version of EPUB 3.1 internally at Wiley journals, for example. Please realize that it is challenging to get a full view of the entire publishing industry.

  - what are the cost metrics for an EPUB update?
  - what's the correct host for such a work?
  - since backward-compatibility's requirements have grown with the
    market, is it still possible to make EPUB evolve?
I believe it is, but we have to gain an understanding of what backward compatibility means and with which systems we will comply. For example, will we retain epub:switch, invented for EPUB 3 but supported almost nowhere or use MathML's native altimage?

  - why is EPUB making any kind of recommendation on content documents?
Going forward, we hoped not to. But, that puts a wrench in your statements about backward compat. Doesn't it?

  - how can we avoid another hiatus of that kind? Should the Charters
    be amended?
  - is the Publishing@W3C architecture efficient?
  - what's the impact on Chairing?
  - how do we define workable exit criteria for specs that do represent
    interoperable implementability and ensure market openness?
As long as we're in W3C, we have the same implementations rules as everyone else (2 implementations).

  - EPUB 3.0.1 is NOT in-scope in the CG's Charter



Tzviya Siegman

Information Standards Lead

Wiley

201-748-6884

tsiegman@wiley.com<mailto:tsiegman@wiley.com>
________________________________
From: Daniel Glazman <daniel.glazman@disruptive-innovations.com>
Sent: Tuesday, December 26, 2017 6:05:41 AM
To: public-publishingbg@w3.org
Cc: Bill McCoy; Jeff Jaffe; Tim Berners-Lee; eb2m-mrt@asahi-net.or.jp
Subject: Deep concerns about the future of EPUB


+----------------------------------------------------------------+
|TL;DR: the future of EPUB is a fiasco of XHTML2 magnitude.      |
|       Urgent actions required.                                 |
+----------------------------------------------------------------+


During the discussions about EPUB 3.1 and its adoption process, two sets
of comments were ignored, or at best severely underestimated:

  - yours truly said the industry was barely starting to convert the
    masses of EPUB 2.x to incompatible 3.0.1 and releasing so quickly
    yet another incompatible 3.1 was a strategic mistake. As an
    IMPLEMENTOR, and probably the only implementor in the world
    releasing EPUB 3.1 in a content editor, I had a lot of reservations
    about the spec despite of its simplifications (in particular in
    its metadata section; the simplifications were technically ok, but
    overkill business-wise). Implementing 3.1 implied yet another
    parser and yet another package processor. It was then
    clear the cost for production chains would not be negligible.

  - the Japanese publishing industry sent a very clear and very
    important letter [1] at the same moment, back in february 2016,
    stating that 3.1 was a no-go for them because of the cost the
    incompatibilities with 3.0.1 would imply. Murata-san recently
    contributed to github issue #982 [2] more arguments, again all very
    clear and, I must say, perfectly understandable.

The adoption process continued on, despite of the warnings, at fast
pace. The speed of adoption of 3.1, added to the lack of implementation
report based on conformant implementations, was also pointed out as a
big issue. No change was implemented, despite of the plausible short-
term merger with W3C.

Less than two years later, the above problems are now blockers:

  - the Japanese industry, the largest 3.0.x adopter in the world,
    clearly rejects adoption of 3.1, with good reasons.

  - if the EPUB ecosystem is anemic because of its complexity, the
    3.1 ecosystem is, as predicted, infinitesimal. Almost no software
    chain widely available in the wild implements it. As far as I
    can tell, less than 1% of BlueGriffon EPUB users create and edit
    3.1. More than 99% use it for 2.x and 3.0.1.

  - the simplifications of 3.1 over 3.0.1 were clearly not enough to
    justify a new version of the spec - and the migration costs - so
    quickly.

Despite of the above, the Publishing activity at W3C recreated more or
less the working habits of IDPF inside W3C. We are already working on
a successor to EPUB 3 that will for sure suffer from the same
ailments. That work is done inside a Working Group but there are
extremely diverging opinions about what should be the next version of
EPUB. Dissonant voices are, for the same reasons and the same way as
before, neglected and the work is done at extremely fast pace, in a
process that I call suboptimal. Backward-compatibility has never been
a first-class element in the standardization process despite of being
a notable highlight in the specs. I have always said, with my
implementor's hat on, that the backward-compatibility of
the EPUB spec ecosystem just does not exist. I can technically prove
this.

EPUB3 (hear 3.1) is maintained by a Community Group. We now see that
the gordian knot of EPUB is not v4 but v3. v3 is where the biggest
effort matters the most. The whole architecture of the Publishing
activity at W3C, with its CG, WG, BG and Steering Committee, is not
tailored to meet the Industry's needs. The BG and the Steering
Committee, « ensuring that the interests of the publishing industry are
appropriately reflected in such publishing activity work », are the
children of the committee that pushed 3.1 so fast despite of the
aforementioned warnings. The BG is in effect leading the CG and
took "a lead role in the process of developing (the) charter" of the
WG. We now see that the whole thing is not in line, at all, with the
largest EPUB industry in the world.

In that light, merging with W3C was seen, at least by me, as a way
of preventing such catastrophic errors to happen again. Formal
Objections and the whole W3C Process could vastly help. The creation of
the BG and its clear handhold on the standardization groups of the
Publishing domain at W3C, were a signal in the other direction. The
creation of the Groups was again done at fast pace, in a suboptimal
process, only to make sure the Groups were created before a Publishing
Summit.

The result is a incredible capharnaüm:

  - the last version of EPUB sees no adoption. Its ecosystem is
    virtually non-existent. Seen from here, EPUB is a dead end.

  - each version of EPUB is incompatible with others. Again, I can
    technically prove this.

  - the technical level of the specifications is, and I say this
    with a W3C dinosaur's eye, low. EPUB is a weak spec that is not
    meeting the quality and implementation constraints levels of other
    Standard Bodies.

  - ISO/IEC adoption of 3.1 would make 3.0.x obsolete

  - Publishing@W3C is not meeting Industry's needs

  - the WG deals with the future of EPUB while the CG maintains the
    existing line of EPUB. And the top of the existing line is now
    rejected by one of our largest user. We now see the most important
    bit here is not the future but the present. As said before, the EPUB
    specs line changes FAR TOO FAST for the Industry. As a result, the
    architecture of the Publishing activity is wrong: the WG should deal
    with 3.x, allowing the full power of W3C Process to apply, while the
    CG should deal with EPUB 4 and WP that are still a topic of complex
    and totally unstabilized discussions.

  - I hope it's now totally out of question to do work in the Publishing
    activity without serious exit criteria based on implementation
    reports. And when I say "implementations", I mean software delivered
    to masses passing a test suite, not prototypes of limited spread or
    lists of users... For the time being, the exit criteria from
    Publishing@W3C are clearly not at W3C level.

As the sole implementor of 3.1 in an editing environment, I am then
asking the Publishing BG to urgently clarify the situation through a
deep self-criticism loop. Implementations of EPUB are EXTREMELY
expensive and it's now out of question to continue moving on without
a clear view, based on an thorough analysis of requirements and facts,
of where we're coming from and where we're going to. It's pointless
to keep discussing 3.2, 4, WP until this is done because the future
is, for the time being, totally uncertain. The architecture of the
Publishing activity at W3C should be reviewed, and modified. The goals
of that Publishing activity, namely EPUB 4, should be reviewed and
probably deferred. WP, and their disruptive changes to the technical
aspects of our industry, are probably not what the market needs right
now.

We don't need a new Publishing Summit. We need a Future of EPUB
face-to-face meeting, with 3 days of open intense discussions, no
panels, no conference, no blah-blah and I think we need it as soon as
possible. We need consensus, good handling of objections, W3M
supervision and strict W3C Process.

Last but not least, the first mission of the self-criticism loop
requested above should be to make sure such a mess never happens
again. As a software vendor, I am shocked by the current situation
and what it potentially implies on the future evolutions of our
software and the finances behind that. In particular, the letter from
the Japanese publishing industry should have reverberated, back in
february 2016, as a formal showstopper.

We cannot afford another fiasco of that magnitude, and we need to fix
the consequences of this one:

  - why is 3.1 such a failure?
  - why didn't the understandable warnings above serve as showstoppers?
  - conclusions on IDPF standardization Process?
  - what is the next level of EPUB and when is it relevant to
    publish it, industry-wise? EPUB 3.0 was released in October 2011
    and more than six years later, the Industry is NOT ready to move
    to the next version, 3.1, that is almost not implemented at all;
    so does it make any sense to start work on another version?
  - what are the cost metrics for an EPUB update?
  - what's the correct host for such a work?
  - since backward-compatibility's requirements have grown with the
    market, is it still possible to make EPUB evolve?
  - why is EPUB making any kind of recommendation on content documents?
  - how can we avoid another hiatus of that kind? Should the Charters
    be amended?
  - is the Publishing@W3C architecture efficient?
  - what's the impact on Chairing?
  - how do we define workable exit criteria for specs that do represent
    interoperable implementability and ensure market openness?
  - EPUB 3.0.1 is NOT in-scope in the CG's Charter
  - etc.

[1]
https://docs.google.com/document/d/1LDmzu7HkFLqXqj-pO5Yx_CVVfEXGz8u81cf1pyCS7gk
[2] https://github.com/w3c/publ-epub-revision/pull/982

</Daniel>