Re: Getting from "Good Start" to "Great Solution” . . ?

Appropriate and timely email and lots of good points! We are aware of
some of the pain points.

This Friday we will be talking about XProc vnext at
http://lanyrd.com/2013/xprocathon/ at XML Prague preconf

I will be on XProc IRC then as well … might be useful to have more of
a real time chat at some point  ?

J

On Wed, Feb 6, 2013 at 12:21 PM, Christopher.R.Ball
<christopher.r.ball@gmail.com> wrote:
> [An Open Letter]
>
>     I have wondered for some time if the xProc community has the will to
> take xProc from a "Good Start" to a "Great Solution” . . . from a “niche
> utility” of the XML elite, to a “fundamental tool” of the masses?
>
>     I use xProc on a regular basis and am grateful for its existence (along
> with Calabash). But every time I use it, I am inevitably frustrated and thus
> reminded of how unlikely it is that it will ever get wide spread adoption.
>
> "Significant Barrier to Entry"
>
>     I was particularly disheartened, when I witnessed a group consider xProc
> as a guiding specification for a new open source software project, only to
> then dismiss it as too complex. (The project, Hydra, is an implementation
> for the highly scalable pipelining of content, as an alternative to the
> batch processing model of hadoop.)
>
>     I can empathize with this perception of “excessive complexity”. But
> based on my own journey with xProc, I would suggest that what a new comer
> sees as complexity is a byproduct of three systemic problems:
>
>         1.    Un-intuitive naming
>         2.    Un-intuitive syntax
>         3.    Insufficient flexibility
>
>     Fred Brooks, in a seminal article (and subsequent book), outlined the
> concepts of accidental versus essential complexity.
>
>         •    Accidental complexity - relates to problems that we create on
> our own and which can be fixed
>         •    Essential complexity - is caused by the problem to be solved,
> and nothing can remove it
>
>     In the case of xProc 1.x, it feels like there is an abundance of small
> accidental forms of complexity. While each instance of accidental complexity
> is seemingly trivial on its own, the cumulative effect is a significant
> barrier to entry. Thus in its current form, xProc feels like it could
> benefit from a book like Douglas Crockford's JavaScript: The Good Parts to
> weed out these unintuitive and idiosyncratic aspects.
>
>     Before exploring what some of them are, I would remind us that we need
> to consider this issue from the novices or students perspective rather than
> our own. As the Gregor Poyola points out in his guide to complex math (How
> to Solve It), we need to first understand the students confusion, how they
> misinterpret a problem, before we can hope to teach it to them.
>
> "Un-intuitive Naming"
>
>     Getting off the ground with a new programming language is challenging
> and I credit the working group for crafting a good start to naming. But the
> semantics need to improve, it is critical that terms be expressive and
> distinct and avoid baggage from other domains. Java was well served in early
> adoption by borrowing terminology and syntax from C while feeling the
> freedom to form new constructs and mechanisms. Likewise, xProc should borrow
> concepts and semantics from xPath, XSLT, and the like. At present there is a
> blend of name and concept collision along with
>
>     For example:
>
>         •    “view-port” as a step name is not consistent with naming that
> clearly conceptually relates from numerous other programming languages. A
> far more consistent and intuitive name would have been “for-each-node” or
> “for-each-fragment” or “for-each-match”.
>         •    “identity” is essentially a “copy” and should be named as such
>         •    “source” & “result” – as a port name for a pipelining language
> which supports recursive structures this is borrowing from the wrong
> paradigm and are overloaded terms as well. The terminology could have been
> better borrowed from the father of pipelines, Petri Nets or fundamental and
> expressive terminology such as inbound content and outbound content.
>
> "How to Continue the Evolution"
>
>     To keep this email focused, I will defer the discussion of the other two
> dimensions troubling xProc to another email (un-intuitive syntax and
> insufficient flexibility). Rather I would like to turn the conversation to a
> process that could assist in continuing xProc’s evolution.
>
>     Grady Booch has been credited with saying, “software [or standards]
> usually does not amount to much until it has been rewritten at least 3
> times.” The challenge for a standard is instituting effective mechanisms
> that effectively foster the:
>
>         a)    identification of impediments
>         b)    discussion of creative solutions
>         c)    assessment of the merits of proposed changes
>
>     In software development user stories can be effective technique for
> surfacing what capabilities are needed. With a standard such as xProc,
> recipes of how to solve commonly reoccurring problems could serve the same
> function.
>
>     The recipes could enable 4 critical pieces to the language's
> development:
>
>         1. What are the problems it is envisioned to solve
>         2. How does it solves those problems (and there may be more than one
> way)
>         3. What are the most important or most common use cases / stories
>         4. How succinctly, intuitively, and efficiently it solves them
>
>     While number 3 has a large subjective element, I think it is the most
> important and yet is absent from the current conversation.
>
>     A healthy discussion/debate of these recipes could lead to the evolution
> of a language. It would be a shame to see xProc languish as the niche tool
> of elite XML jocks rather than as a potential draw to XML itself. But that
> means a willingness to really look at xProc with a critical eye and a
> willingness to change what at first glance may seem embedded in concrete.
>
>     Given the relative nascence of xProc, I send this email in the hope of
> encouraging the community to contend with these issues now rather than let
> it wither in a niche of obscurity as other well intended standards have.
>
>     While I certainly do not have all the answers, I do hope I can stimulate
> the community and assist in evolving xProc to realize its full potential.
>
> Regards,
>
> Christopher
>
>
>

Received on Wednesday, 6 February 2013 11:28:47 UTC