Let's consider the requirement for parameters and some use cases.

The requirement: in the simple case where the pipeline author does
nothing with (may even be entirely ignorant of) parameters, users can
still pass parameters to steps in the pipeline.

In other words, given this pipeline:

<p:declare-step version="1.0" name="main" type="ex:simple"
  <p:input port="source"/>
  <p:output port="result"/>


    <p:input port="stylesheet">
      <p:document href="style.xsl"/>

It must be possible for users to pass parameters to the contained XSLT
step. This is the requirement that forces us to indulge in some sort
of magic. The whole magic binding of parameter input ports in V1 is
based on this requirement.

If we abandon this requirement, then a solution like the one that
Vojtech proposes becomes the only option available to the pipeline
author and there is no magic.

It also means that the vast majority of pipelines will *not* accept
*any* parameters because the pipeline author will never have thought
of it.

  There are are some things that bother me about Vojtech's proposal,
  including one substantive error. Vojtech proposes:

    <p:input port="source"/>
    <p:output port="source"/>
    <p:option name="xquery-params" as="map:map(xs:QName, item()*)?"/>
    <p:option name="xslt-params" as="map:map(xs:QName, item()*)?"/>

  All well and good so far, but analyzing that pipeline and working
  out what the valid command line options may be looks pretty
  daunting. Made even more daunting by the fact that option names are
  QNames. I don't doubt that it could be done, but it looks pretty

  The Edinburgh proposal says that there are named hashmaps that come
  from the outside environment. No pipeline inspection is required to
  determine what a valid command line (or API) would be.

    <p:xquery parameters="$xquery-params">

  And you can't do that. Even assuming the AVT "{$xquery-params}" was
  intended, it doesn't work. AVTs have to produce single (untypedAtomic)
  value. If you want to pass a map to a step, you have to do it the
  long way:

      <p:with-option name="parameters" select="$xquery-params"/>

One feature of the Edinburgh proposal is that, by default, all steps
at all levels in all pipelines get the "unnamed" bag of parameters by

That does not mean that pipeline authors have no control, it just
means that you have to take extra steps if you want to exert control.

Here is a pipeline that demonstrates control:

<p:declare-step type="ex:step">
  <p:input port="source"/>
  <p:output port="source"/>
  <p:option name="xquery-params" as="map:map(xs:QName, item()*)?"
  <p:option name="xslt-params" as="map:map(xs:QName, item()*)?"

     <p:with-option name="parameters" select="$xquery-params"/>

     <p:with-option name="parameters" select="$xslt-params"/>

Left to its own devices, for example, if it's the top-level pipeline,
the "unnamed" parameters will go to the XSLT step and the "xquery"
parameters will go to the XQuery step. (I could just as easily have
used two different names, but I, as the pipeline author, think that
passing parameters to XSLT is more common than XQuery so I've made
that easy.)

If I incorporate that pipeline into *my* pipeline as a third party
library, I have complete control over what parameters get passed
through. Perhaps I want to make it easy to control the XQuery step
but impossible to fiddle with the parameters in the XSLT step.
Easy peasy:

    <p:with-option name="xquery-parameters" select="p:parameters()"/>
    <p:with-option name="xslt-parameters" select="map:new()"/>

Perhaps I want to use different parameter set names. Equally easy:

    <p:with-option name="xquery-parameters" select="p:parameters('xqy')"/>
    <p:with-option name="xslt-parameters" select="p:parameters('xsl')"/>

Obviously I can construct any parameter map I wish, through any means
at my disposal.

The only thing you don't get, really, is the ability to control the
parameters used by a third party pipeline that was written without
thinking about parameters.

In fact, that is, I think, the only thing we've lost.

Going back to the ex:simple pipeline above, there's nothing the caller
can do to change which parameters are used by that pipeline.

I think the 80% case, maybe the 95% case, is that no parameters are
passed to the pipeline *at all*. In which case, it doesn't matter what
control we could have had.

You might think that we've also lost the ability to distinguish which
parameters go to which steps. If pipeline authors don't think about
parameters, then all steps that accept parameters get the same

That's actually the case even in V1.0. The single primary parameter
port feeds all the steps by default. As above, most of the time there
are no parameters and it doesn't matter.

In fact, it only matters in the single case where there are two or
more steps that accept parameters in the pipeline and they have
parameter names that collide, *and*, someone has, in practice,
specified a value for one of the colliding parameters.

On balance, and considering the *enormous* syntactic and semantic
simplifications it carries with it, I think the Edinburgh proposal is
"good enough".

                                        Be seeing you,

Norman Walsh
Lead Engineer
MarkLogic Corporation
Phone: +1 512 761 6676

Received on Wednesday, 5 March 2014 23:08:07 UTC