Re: Difficulty overriding parameter values

I received the following request from alex.g.muir@gmail.com:

  I was looking over your posts at the following link regarding Xproc 
and using and xml file
  to drive the processing. I'm looking to do something similar I was 
wondering if you could
  share some solution on the xproc dev list to show how you ended up 
solving what you were doing.

  http://lists.w3.org/Archives/Public/xproc-dev/2009Jul/att-0013/00-part

So here goes:
=========================================================
It's still a work in (intermittent) progress, but what I currently do is 
use the http://www.w3.org/ns/xproc-step param-set element for making 
configuration files. For example:

---------------------
<c:param-set xmlns:c="http://www.w3.org/ns/xproc-step">

  <!-- run this pipeline... -->
  <c:param name="pipeline" value="xyz"/>

  <!-- with these parameters... -->
  <c:param name="input-file" value="file:///some/input/file.ext"/>
  <c:param name="output-file" value="file:///some/output/file.ext"/>
  <c:param name="abc" value="def"/>

</c:param-set>
-----------------------

A configuration file always contains one "master" parameter, "pipeline" 
that names the actual pipeline to be run. All the other parameters are 
the parameters required by that pipeline.

This parameter file is submitted as the only input to a master pipeline 
(called run-pipe.xpl) which just contains a big choose element that runs 
the specified pipeline and passes on all the specified parameters.

---------------------
<p:declare-step xmlns:p="http://www.w3.org/ns/xproc"
                xmlns:c="http://www.w3.org/ns/xproc-step"
                xmlns:pipes="http://xmlns.escenic.com/2009/xproc/my-pipes"
                xmlns:lib="http://xmlns.escenic.com/2009/xproc/my-library"
                name="run-pipe">

      <p:input port="source"/>
      <p:input port="par" kind="parameter"/>

      <!-- output port not needed, but this will output the URL of the 
saved file -->
      <p:output port="result">
        <p:pipe step="end-of-pipe" port="result"/>
      </p:output>

  <p:import href="pipes.xpl"/>

  <p:variable name="pipeline" 
select="/c:param-set/c:param[@name='pipeline']/@value">
    <p:pipe step="run-pipe" port="par"/>
  </p:variable>

  <p:choose>
    <p:when test="$pipeline='abc'">
      <pipes:abc name="abc">
        <p:input port="par" kind="parameter">
          <p:pipe step="run-pipe" port="par"/>
        </p:input>
      </pipes:abc>
    </p:when>
    <!-- ...lots more "whens"... -->
   <p:otherwise>
      <p:identity>
        <p:input port="source">
          <p:inline>
            <ebd:message>Pipe not found</ebd:message>
          </p:inline>
        </p:input>
      </p:identity>
    </p:otherwise>
  </p:choose>

  <p:identity name="end-of-pipe"/>

</p:declare-step>
---------------------

pipes.xpl contains all the actual pipelines, each of which has the 
following basic structure:

---------------------
  <p:declare-step type="pipes:abc" name="pipe-abc">

        <p:input port="source"/>
        <p:input port="par" kind="parameter"/>
        <p:output port="result">
          <p:pipe step="store" port="result"/>
        </p:output>

    <p:import href="library.xpl"/>

    <p:variable name="input-file" 
select="/c:param-set/c:param[@name='input-file']/@value">
      <p:pipe step="pipe-abc" port="par"/>
    </p:variable>
    <p:variable name="output-file" 
select="/c:param-set/c:param[@name='output-file']/@value">
      <p:pipe step="pipe-abc" port="par"/>
    </p:variable>

    <p:load name="load">
      <p:with-option name="href" select="$input-file"/>
    </p:load>

    <lib:some-basic-pipe name="some-basic-pipe">
      <p:input port="par" kind="parameter">
        <p:pipe step="pipe-abc" port="par"/>
      </p:input>
    </lib:some-basic-pipe>

    <!-- ...more pipes from library.xpl... -->

    <p:store name="store">
      <p:with-option name="href" select="$output-file"/>
    </p:store>

  </p:declare-step>
---------------------

The pipelines in pipes.xpl extract the parameters they need to "load" 
input files and "store" output files, and otherwise just call a sequence 
of simpler pipelines from library.xpl.

The pipelines in library.xpl are mostly just sequences of xslt and other 
atomic steps:

---------------------
  <p:declare-step type="lib:some-basic-pipe" name="lib-some-basic-pipe">
        <p:input port="source"/>
        <p:input port="par" kind="parameter"/>
        <p:output port="result">
          <p:pipe step="trf2" port="result"/>
        </p:output>
   
    <p:xslt name="trf1">
      <p:input port="stylesheet">
          <p:document href="xsl/trf1.xsl"/>
      </p:input>
    </p:xslt>

    <p:xslt name="trf2">
      <p:input port="stylesheet">
        <p:document href="xsl/trf2.xsl"/>
      </p:input>
    </p:xslt>
  </p:declare-step>
---------------------
 
All the parameters specified in the configuration file are passed in to 
all the transformations, which can use the ones they need. trf1.xsl, for 
example, might contain:

---------------------
    <xsl:param name="abc"/>
---------------------

and would therefore use the setting 'def' specified in the configuration 
file.

This structure means you can build up a library.xpl containing basic 
pipes that can be re-used in various ways, and a higher-level pipes.xpl 
that slots them together in more complex ways and deals with reading and 
writing files. It's all accessible in a uniform way - one pipe to call 
them all, plus a configuration file. Since I'm using the standard 
http://www.w3.org/ns/xproc-step param-set element for the configuration 
file, you can alternatively specify parameters on the command line.

In order to make this framework really useable, two things are needed:

- strict naming conventions for parameters
- an automated means of exposing all the parameters used by a particular 
pipeline.

What I am thinking of doing is defining a standard way of documenting 
all parameters, wherever they appear - in XSLT transformations or pipes 
- most likely using a namespace. It would then be possible to make an 
XSLT transformation that, given the name of one of the top-level pipes 
in pipes.xpl, could drill down through all the called sub-pipes and 
transformations, find all the parameters used and generate a complete, 
fully-documented configuration file.

Any advice, suggestions, offers of help etc. are more than welcome.

If this list is an inappropriate forum for this kind of discussion (most 
posts seem to be focused on the specification, rather than 
applications), maybe somebody can suggest an alternative?

Regards,

Kevin Flynn

Received on Friday, 27 November 2009 08:51:52 UTC