Re: Where's the parallelize step?

On Mon, Apr 20, 2009 at 2:01 PM, Henry S. Thompson <ht@inf.ed.ac.uk> wrote:
> -----BEGIN PGP SIGNED MESSAGE-----
> Hash: SHA1
>
> Costello, Roger L. writes:
>
>> Why doesn't XProc have a parallelize step? That is, a step that
>> enables two subpipelines to proceed in parallel. Is there any
>> discussion of adding a parallelize step to XProc?
>
> If I've understood you correctly, the answer is "because it doesn't
> need one".  The semantics of XProc do not require the evaluation of
> the steps in a pipeline to be any more serialised than their explicit
> dependencies require.  So it's open to implementations to parallelise
> as much as they like/can.
>
> I have in the past used the following as a sort of _aide memoire_:
>
>  It should be possible to implement XProc by starting separate
>  threads for _every_ step in the controlling pipeline, and letting
>  input/output/parameter ports control the actual order of execution.
>
> I believe it is
>
>  a) still the case that the above will work;
>
>  b) implicit in the above that if you have multiple processors, you
>     will get parallel execution where the above story allows for it.
>
> There is at least one case where a smart implementation can
> parallelise which the above would not immediately uncover.  The
> execution of the sub-pipeline of a p:for-each should in principle be
> parallelisable across the different inputs to the p:for-each (provided
> they have no side-effects).

this is a nice description and perhaps section H. could paraphrase
more this pov.

> In any case, I think we have all the room we need to explore this
> space w/o any explicit steps, but maybe James's archive search will
> uncover something I've missed. . .

I think my point was that there maybe scenarios where we want to
explicitly say 'serialize' or 'parallize' but I was convinced at the
time that an extension attribute will cover this off instead of
building it into the core ... which I am more then happy with.

Roger, feel free to cmon over to http://exproc.org/ as well.

cheers, J

Received on Monday, 20 April 2009 13:20:10 UTC