Re: State of XProc: language and community

One thing I wonder about is the memory limitations in Calabash and the whole
concept of being able to save and use data from any step that occurred
previously; If they are related and if that requirement has any effect on an
efficient means to implement getting some data that is processed elsewhere?

Would an xproc that forced the developer to store data in some fast data
storage in memory server and thus put,get,wait for data make for a much
easier to develop straight forward low memory use pipeline from a
programming perspective?

The files that are available on the data server could be accessed by some
associated name or id or kept in different structures like a stack or queue.
Red black tree based on some sort from xslt/xpath and easily accessed by any
pipeline with some varying get calls depending on the situation...

<get position="last()" storageList="FileQueueOne"/>
<get position="#TOP" storageList="FileStackABC"/>

Mainly I'm wondering if restricting the pipeline language to work like that
would make any difference to the efficiency of the software written to
handle the spec? Perhaps the spec allows for such an implementation already
however I do wonder if the language would become more simplified.

I think what has been developed thus far is great btw. I use xproc every day
in my work and I like it and have high regards for those supporting it.
Certainly I hold the view that xproc is great generally.

Thanks

On Sat, Mar 5, 2011 at 8:19 PM, Christopher.R.Ball <
christopher.r.ball@gmail.com> wrote:

>  Tony,
>
> For what it's worth - I concur with many of your points.
>
> I written numerous extensions, crafted visual illustrations, some cookbook
> like features, and written about several opportunities for improvement . . .
> but gave up with similar frustrations to yours.
>
> Personally, I have been contemplating creating my own site (not to grab the
> stiring wheel, but) to get something that I could iterate over and better
> guide and share with those that  I collaborate with. A handful of us have
> been toying with the idea of written our own 'wrapper' as it were to better
> deal with some of the more painful idiosyncrasies. Though we realize that
> many may consider that sacrilege, we are just striving for something just a
> bit more pragmatic, efficient, and simplistic to use.
>
> All the best,
>
> Christopher
>
>
>
> On 3/5/2011 2:38 PM, Tony R. wrote:
>
> I compiled a few thoughts I have had over and over again recently.  I’m
> sharing them with the list as feedback for the entire XProc ecosystem.
>
>  Do with the feedback what you will.  Flame me.  Delete it.  Respond.
>  Pass it onto others.  If nobody ever reads this message, at least I wrote
> it down so that I can get it out of my head.  ☺
>
>  In any case, I welcome feedback and discussion on any of this.  Don't be
> shy!  ;-D
>
>
>  *XProc LANGUAGE*
>
>  *Needs better separation.*
>
>  There's a striking lack of separation between the plethora of things
> that (may or may not) appear at the beginning of a step—such as ports,
> options, and so on—and sub-pipelines and contained steps.
>
>
>  Maybe I'm just used to the head/body pattern from HTML.  But I can't help
> but imagine that something similar might be beneficial for XProc.
>
>  *
> *
>
> *Needlessly redundant steps.*
>
>  Many separate p:validate-with-* steps exist when a single p:validate step with
> something like @with="relax-ng"  would suffice. This needlessly adds to
> language bloat IMO.
>
>  *
> *
>
> *Lack of syntactic sugar.*
>
>  There are *so *many times when the most common use case has to be written
> and re-written over and over again.  This drives me absolutely crazy and *
> significantly* increases the size of my code. The XProc wiki (
> http://wiki.XProc.org/XProcVNext) outlines this—and a number of other
> syntactic sugar enhancements—that would make XProc so much more *pleasant*.
>
>
>  *
> *
>
> *Data types.*
>
>  XProc’s lack of built-in data types can be unbelievably frustrating.  And
> XML is *designed* for modular reuse—including plenty of data types that
> have been battle-tested for years.  If XProc supported even just a tiny
> subset of these, it would make life *so* much easier.
>
>
>  At the very least, I think XProc should know the difference between a
> document, a URI, and a string.
>
>
>  While we're at it, it would be extra awesome if data types were usable
> directly inside attributes instead of  having to nest like <p:with-option
> … />. This could easily be accomplished without ambiguity for
> processors by making use of XPath's built in functions, e.g. string(‘aaa’),
> base-uri(‘http://weee/subdir/‘), and so on.  (C<http://www.w3.org/TR/xpath-functions/#constructor-functions-for-xsd-types>onstructor
> Functions for XML Schema Built-In Types seems worth a quick review here.)
>
>
>
>  *XProc COMMUNITY*
>
>  *Documentation.*
>
>  I love Calabash!, …but the lack of documentation is a real downer.  It
> even resulted in quite a shock for me when I discovered there was a
> mechanism for providing Calabash with custom settings<http://norman.walsh.name/2009/07/22/xmlCatalogsandXProc>.
>  I’d been using Calabash for months at the time, and had no idea this was
> even possible.
>
>  It would also be great if there was something with the same role as
> Javadoc for XProc.  (I.E., Parse XProc files and output human-friendly
> documentation in XHTML [or whatever]).
>
>  Documentation—that is, *good* documentation—unites people and fosters
> collaboration.  We need more of it.
>
>
>  *Un-unified community efforts.*
>
>  One of my favorite things about XML community is how active it is.
>  Members are very supportive each other.  And there seems to be a delightful
> absence of the “elitist jerk programmer” stereotype mindset.  If I didn’t
> know any better, I might think most of us really love this stuff, and enjoy
> that others love it, too!  ;-)
>
>  However, although the discussion lists are quite healthy, solutions are
> scattered. People come up with great solutions…and they might tell the
> mailing list about them.  They might even post them on the web.  *But
> there is no central place to share solutions.*
>
>  Common recurring problems may have a dozen independently-created
> solutions to them, all created by different people.  At the same time, other
> programmers may not be as skilled, and become frustrated in searching for a
> solution and give up.
>
>  *For example:*
>
>   Norm’s online XProc Book <http://xprocbook.com/book/book-1.html> has a
> fully-functional ex:recursive-directory-list<http://xprocbook.com/book/refentry-61.html> step.
>  It’s written in XProc itself, so it’s even processor-independent.  Cool!
>
>
>    …So, why isn’t this brilliance on EXProc.org?
>
>
>  The closest thing to a central place for this stuff is the XProc wiki<http://wiki.xproc.org>…but
> activity remains almost nonexistent here.
>
>  The EXPath Project <http://www.expath.org> also aims to be a central
> space for collaboration.  It embodies my vision of what the XML community
> needs most.  …But although it’s slightly more populated with content than
> the XProc wiki, it appears to have been on hiatus for at least a couple of
> years.  (I’m trying to begin contributing to this project in the hopes of
> getting some momentum going again.  Fingers crossed!)
>
>  Seriously, everybody: we have an *amazing*, *brilliant*, *inventive*, *
> helpful*, and (on mailing lists at least) *active* community.  We just
> need to focus all that talent on *pooling our resources!*
>
>  It would be so very, very glorious.
>
>
>  *Norm.*
>
>  I love you Norm!  (strictly professionally of course! ☺)  I think that if
> anyone else attempted to do as much as you, their head would probably
> explode.
>
>
>  However, there are have been many occasions—far too numerous to
> recall—where I wanted to contribute something to the websites for XProc,
> Calabash, and EXProc…but I wasn't able to.
>
>
>   *Aside to Norm:*
>
>    I *did* try writing you personally, writing the mailing list, and even
> leaving a note for you on the XProc wiki hoping you might notice.  Alas, no
> such luck.  I figured it would take more than that for little ol’ me to
> register on your very busy radar, but I had to try.
>
>
>  No hard feelings!  Just a disclaimer. ☺
>
>
>  …Meanwhile, the wiki goes pretty much untouched.  So there’s a wiki with
> maybe one page of useful info that is dead in terms of activity, and there’s
> plenty of useful pages on the xproc.org / xmlcalabash.org / exproc.org sites,
> but they are pretty much dead in terms of activity (with the exception of
> Calabash releases).
>
>
>  I really wish there was a way this stuff into the hands of our awesome
> community…but right now there is no such space.  ☹
>
>
>
> —Tony
>
>
>


-- 
Alex
-----
Currently:
Freelance Software Engineer 6+ yrs exp
<http://www.facebook.com/pages/Bafila/125611807494851>
Previously:
https://sites.google.com/a/utg.edu.gm/alex/


A Bafila, is two rivers flowing together as one:
http://www.facebook.com/pages/Bafila/125611807494851