On Fri, Nov 27, 2009 at 8:51 AM, Kevin Flynn <kevin@escenic.com> wrote:

I received the following request from alex.g.muir@gmail.com:

I was looking over your posts at the following link regarding Xproc and using and xml file
to drive the processing. I'm looking to do something similar I was wondering if you could
share some solution on the xproc dev list to show how you ended up solving what you were doing.

http://lists.w3.org/Archives/Public/xproc-dev/2009Jul/att-0013/00-part

So here goes:
=========================================================
It's still a work in (intermittent) progress, but what I currently do is use the http://www.w3.org/ns/xproc-step param-set element for making configuration files. For example:

---------------------
<c:param-set xmlns:c="http://www.w3.org/ns/xproc-step">


<c:param name="pipeline" value="xyz"/>


<c:param name="input-file" value="file:///some/input/file.ext"/>
<c:param name="output-file" value="file:///some/output/file.ext"/>
<c:param name="abc" value="def"/>

</c:param-set>
-----------------------

A configuration file always contains one "master" parameter, "pipeline" that names the actual pipeline to be run. All the other parameters are the parameters required by that pipeline.

This parameter file is submitted as the only input to a master pipeline (called run-pipe.xpl) which just contains a big choose element that runs the specified pipeline and passes on all the specified parameters.

---------------------
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc"
xmlns:c="http://www.w3.org/ns/xproc-step"
xmlns:pipes="http://xmlns.escenic.com/2009/xproc/my-pipes"
xmlns:lib="http://xmlns.escenic.com/2009/xproc/my-library"
name="run-pipe">

<p:input port="source"/>
<p:input port="par" kind="parameter"/>


<p:output port="result">
<p:pipe step="end-of-pipe" port="result"/>
</p:output>

<p:import href="pipes.xpl"/>

<p:variable name="pipeline" select="/c:param-set/c:param[@name='pipeline']/@value">
<p:pipe step="run-pipe" port="par"/>
</p:variable>

<p:choose>
<p:when test="$pipeline='abc'">
<pipes:abc name="abc">
<p:input port="par" kind="parameter">
<p:pipe step="run-pipe" port="par"/>
</p:input>
</pipes:abc>
</p:when>

<p:otherwise>
<p:identity>
<p:input port="source">
<p:inline>
<ebd:message>Pipe not found</ebd:message>
</p:inline>
</p:input>
</p:identity>
</p:otherwise>
</p:choose>

<p:identity name="end-of-pipe"/>

</p:declare-step>
---------------------

pipes.xpl contains all the actual pipelines, each of which has the following basic structure:

---------------------
<p:declare-step type="pipes:abc" name="pipe-abc">

<p:input port="source"/>
<p:input port="par" kind="parameter"/>
<p:output port="result">
<p:pipe step="store" port="result"/>
</p:output>

<p:import href="library.xpl"/>

<p:variable name="input-file" select="/c:param-set/c:param[@name='input-file']/@value">
<p:pipe step="pipe-abc" port="par"/>
</p:variable>
<p:variable name="output-file" select="/c:param-set/c:param[@name='output-file']/@value">
<p:pipe step="pipe-abc" port="par"/>
</p:variable>

<p:load name="load">
<p:with-option name="href" select="$input-file"/>
</p:load>

<lib:some-basic-pipe name="some-basic-pipe">
<p:input port="par" kind="parameter">
<p:pipe step="pipe-abc" port="par"/>
</p:input>
</lib:some-basic-pipe>



<p:store name="store">
<p:with-option name="href" select="$output-file"/>
</p:store>

</p:declare-step>
---------------------

The pipelines in pipes.xpl extract the parameters they need to "load" input files and "store" output files, and otherwise just call a sequence of simpler pipelines from library.xpl.

The pipelines in library.xpl are mostly just sequences of xslt and other atomic steps:

---------------------
<p:declare-step type="lib:some-basic-pipe" name="lib-some-basic-pipe">
<p:input port="source"/>
<p:input port="par" kind="parameter"/>
<p:output port="result">
<p:pipe step="trf2" port="result"/>
</p:output>
<p:xslt name="trf1">
<p:input port="stylesheet">
<p:document href="xsl/trf1.xsl"/>
</p:input>
</p:xslt>

<p:xslt name="trf2">
<p:input port="stylesheet">
<p:document href="xsl/trf2.xsl"/>
</p:input>
</p:xslt>
</p:declare-step>
---------------------

All the parameters specified in the configuration file are passed in to all the transformations, which can use the ones they need. trf1.xsl, for example, might contain:

---------------------
<xsl:param name="abc"/>
---------------------

and would therefore use the setting 'def' specified in the configuration file.

This structure means you can build up a library.xpl containing basic pipes that can be re-used in various ways, and a higher-level pipes.xpl that slots them together in more complex ways and deals with reading and writing files. It's all accessible in a uniform way - one pipe to call them all, plus a configuration file. Since I'm using the standard http://www.w3.org/ns/xproc-step param-set element for the configuration file, you can alternatively specify parameters on the command line.

In order to make this framework really useable, two things are needed:

- strict naming conventions for parameters
- an automated means of exposing all the parameters used by a particular pipeline.

What I am thinking of doing is defining a standard way of documenting all parameters, wherever they appear - in XSLT transformations or pipes - most likely using a namespace. It would then be possible to make an XSLT transformation that, given the name of one of the top-level pipes in pipes.xpl, could drill down through all the called sub-pipes and transformations, find all the parameters used and generate a complete, fully-documented configuration file.

Any advice, suggestions, offers of help etc. are more than welcome.

If this list is an inappropriate forum for this kind of discussion (most posts seem to be focused on the specification, rather than applications), maybe somebody can suggest an alternative?

Regards,

Kevin Flynn