Re: Thinking about injection

I would like to be able to inject arbitrary filter steps into a given
pipeline.  For that to work, I need to be able to specify the insertion
location, and preferably this (static) location specification would be
straightforward to deduce from the source code of the pipeline.

One use for injecting into an existing pipeline would be debugging.  For
that to work efficiently, one would have to be able to reference steps to
insert in a concise matter, maybe from a library.

I would also like to be able to trace particular points in pipeline
execution like you've laid out.  In addition to enabling tracing for
particular connections, it would be helpful if there would be a global
execution trace where each connection in the pipeline dumps its contents to
a file that is named after the locator plus dynamic information (i.e. the
number of the document in a sequence).

To me, the injection spec would mostly describe a syntax for locators, both
in the static and the dynamic context of a pipeline, which would then be
used for traces as well as to specify injection or trace points, at the
implementations' discretion.

-Hans

On Tue, Sep 19, 2017 at 10:56 AM, Norman Walsh <ndw@nwalsh.com> wrote:

> Geert Bormans <geert@gbormans.telenet.be> writes:
> > The idea is to have an injection spec We agreed we need it, we
> > committed to work on it soon, but we have not specified a single bit
> > of it, so all of the following is pure speculation
>
> Ok. I’m still trying to get my head around what needs to be specified.
>
> Imagine I have a complicated pipeline that I think contains a bug. The
> goal is to have a way to enable debugging for that pipeline without
> changing it. And the goal of a spec would be to have an interoperable
> way to specify this. (Are we convinced that we need interoperability
> that this level?)
>
> So. I might want to know:
>
> 1. When a particular step runs
> 2. What the computed values of its options are
> 3. What its inputs are
> 4. What its outputs are
>
> Presumably it would be useful to enable these features with
> conditional expressions.
>
> What more would you want?
>
>                                         Be seeing you,
>                                           norm
>
> --
> Norman Walsh
> Lead Engineer
> MarkLogic Corporation
> Phone: +1 512 761 6676
> www.marklogic.com
>



-- 
LambdaWerk GmbH
Oranienburger Straße 87/89
10178 Berlin
Phone: +49 30 555 7335 0
Fax: +49 30 555 7335 99

HRB 169991 B Amtsgericht Charlottenburg
USt-ID: DE301399951
Geschäftsführer:  Hans Hübner

http://lambdawerk.com/