Re: Sub-pipelines

Alessandro Vernet wrote:
> Jeni, Rui, Norm,
> 
> It is a given that we want a pipeline to be able to call another
> pipeline. Now this raises two questions, both of which being discussed
> in this thread:
> 
> 1) Do we want to provide a construction in the pipeline language to
> declare sub-pipelines, that is: a pipeline declared in a pipeline
> document (like some languages allow a function to be declared in a
> function)?
> 
> Norm suggested a way to declare a sub-pipeline. I asked if we really
> need this feature. Rui say he can live with just externally defined
> pipelines. I think that we don't really have a use case or requirement
> that would justifies adding an additional construction to the
> language. So I am not strongly in favor of addressing the issue of how
> to define a sub-pipeline in a pipeline in a first version of pipeline
> language.
> 

Maybe an intermediate solution could be something like pipeline 
aggregation in some root element. This way, we could define several 
pipelines inside the same document without defining sub-pipelines inside 
a pipeline. This may lead to another question: how to trigger the 
execution of one pipeline? Maybe it's up to the environment to figure it 
out, or we can have a special purpose name for the default pipeline 
(like Make's "all", c/c++/java main(), etc.). E.g.:

<p:root>
     <p:pipeline name="main">
         <p:input ... />
         <p:output ... />

         <p:step name="pipeline">
            <p:input name="pipeline" href="#expandandvalidate" />
            ...
         </p:step>
         <p:step name="xslt" ... />
         ...
     </p:pipeline>

     <p:pipeline name="expandandvalidate">
        ...
     </p:pipeline>
</p:root>


> 2) Should invoking a pipeline be any different than invoking, say, an
> XSLT stylesheet? Should we invoke a pipeline with:
> 
> <p:step name="pipeline">
>     <p:input name="pipeline" href="expandandvalidate.xpl"/>
>     <p:input name="schemas" href="a.xsd"/>
>     <p:input name="document" href="mydoc.xml"/>
>     <p:output label="ok1"/>
> </p:step>
> 
> Or:
> 
> <p:call-pipeline name="expandandvalidate">
>     <p:input name="schemas" href="a.xsd"/>
>     <p:input name="document" href="mydoc.xml"/>
>     <p:output label="ok1"/>
> </p:call-pipeline>
> 
> Jeni says that the second way is better because it makes it easier for
> GUI tools and users to identify reusable parts. I tend to like the
> first way, because:
> 
> a) It makes the pipeline language simpler (no need to define anything
> special to call pipeline).
> b) It allows a pipeline to generate another pipeline (say with XSLT)
> and to run it.
> 

We should be careful with dynamically generated pipelines, as that 
feature enables the construction of pipelines with a cyclic digraph 
topology in runtime (i.e. two pipelines invoking each other).

I'm not against cycles, but my feeling is that cycles should be verified 
statically (in compile time).


Rui