Getting from "Good Start" to "Great Solution” . . ?

[An Open Letter]

     I have wondered for some time if the xProc community has the will 
to take xProc from a "Good Start" to a "Great Solution” . . . from a 
“niche utility” of the XML elite, to a “fundamental tool” of the masses?

     I use xProc on a regular basis and am grateful for its existence 
(along with Calabash). But every time I use it, I am inevitably 
frustrated and thus reminded of how unlikely it is that it will ever get 
wide spread adoption.

"Significant Barrier to Entry"

     I was particularly disheartened, when I witnessed a group consider 
xProc as a guiding specification for a new open source software project, 
only to then dismiss it as too complex. (The project, Hydra, is an 
implementation for the highly scalable pipelining of content, as an 
alternative to the batch processing model of hadoop.)

     I can empathize with this perception of “excessive complexity”. But 
based on my own journey with xProc, I would suggest that what a new 
comer sees as complexity is a byproduct of three systemic problems:

         1.    Un-intuitive naming
         2.    Un-intuitive syntax
         3.    Insufficient flexibility

     Fred Brooks, in a seminal article (and subsequent book), outlined 
the concepts of accidental versus essential complexity.

         •    Accidental complexity - relates to problems that we create 
on our own and which can be fixed
         •    Essential complexity - is caused by the problem to be 
solved, and nothing can remove it

     In the case of xProc 1.x, it feels like there is an abundance of 
small accidental forms of complexity. While each instance of accidental 
complexity is seemingly trivial on its own, the cumulative effect is a 
significant barrier to entry. Thus in its current form, xProc feels like 
it could benefit from a book like Douglas Crockford's JavaScript: The 
Good Parts to weed out these unintuitive and idiosyncratic aspects.

     Before exploring what some of them are, I would remind us that we 
need to consider this issue from the novices or students perspective 
rather than our own. As the Gregor Poyola points out in his guide to 
complex math (How to Solve It), we need to first understand the students 
confusion, how they misinterpret a problem, before we can hope to teach 
it to them.

"Un-intuitive Naming"

     Getting off the ground with a new programming language is 
challenging and I credit the working group for crafting a good start to 
naming. But the semantics need to improve, it is critical that terms be 
expressive and distinct and avoid baggage from other domains. Java was 
well served in early adoption by borrowing terminology and syntax from C 
while feeling the freedom to form new constructs and mechanisms. 
Likewise, xProc should borrow concepts and semantics from xPath, XSLT, 
and the like. At present there is a blend of name and concept collision 
along with

     For example:

         •    “view-port” as a step name is not consistent with naming 
that clearly conceptually relates from numerous other programming 
languages. A far more consistent and intuitive name would have been 
“for-each-node” or “for-each-fragment” or “for-each-match”.
         •    “identity” is essentially a “copy” and should be named as such
         •    “source” & “result” – as a port name for a pipelining 
language which supports recursive structures this is borrowing from the 
wrong paradigm and are overloaded terms as well. The terminology could 
have been better borrowed from the father of pipelines, Petri Nets or 
fundamental and expressive terminology such as inbound content and 
outbound content.

"How to Continue the Evolution"

     To keep this email focused, I will defer the discussion of the 
other two dimensions troubling xProc to another email (un-intuitive 
syntax and insufficient flexibility). Rather I would like to turn the 
conversation to a process that could assist in continuing xProc’s evolution.

     Grady Booch has been credited with saying, “software [or standards] 
usually does not amount to much until it has been rewritten at least 3 
times.” The challenge for a standard is instituting effective mechanisms 
that effectively foster the:

         a)    identification of impediments
         b)    discussion of creative solutions
         c)    assessment of the merits of proposed changes

     In software development user stories can be effective technique for 
surfacing what capabilities are needed. With a standard such as xProc, 
recipes of how to solve commonly reoccurring problems could serve the 
same function.

     The recipes could enable 4 critical pieces to the language's 
development:

         1. What are the problems it is envisioned to solve
         2. How does it solves those problems (and there may be more 
than one way)
         3. What are the most important or most common use cases / stories
         4. How succinctly, intuitively, and efficiently it solves them

     While number 3 has a large subjective element, I think it is the 
most important and yet is absent from the current conversation.

     A healthy discussion/debate of these recipes could lead to the 
evolution of a language. It would be a shame to see xProc languish as 
the niche tool of elite XML jocks rather than as a potential draw to XML 
itself. But that means a willingness to really look at xProc with a 
critical eye and a willingness to change what at first glance may seem 
embedded in concrete.

     Given the relative nascence of xProc, I send this email in the hope 
of encouraging the community to contend with these issues now rather 
than let it wither in a niche of obscurity as other well intended 
standards have.

     While I certainly do not have all the answers, I do hope I can 
stimulate the community and assist in evolving xProc to realize its full 
potential.

Regards,

Christopher

Received on Wednesday, 6 February 2013 11:21:33 UTC