Re: PipX, a portable library of XProc pipelines and steps

  Thanks, Jostein.  I gave it a try, and put the resulting HTML
report at http://pipx.org/tmp/pipx-parameter.html.  This is what
I get if I execute the following command:

    calabash \
      -o html=pipx-parameter.html \
      -o result=pipx-parameter-report.xml \
      -o junit=pipx-parameter-junit.xml \
      -i pipx-parameter.xprocspec \
      $(XPROCSPEC)

  But I have serious concerns about the language itself.  I find
it rather verbose, non composable, and it lacks assertions as
crutial as on errors, apparently.  I suffers from the same flaws
as XSpec does (I will not blame you for those, surely :-p).

  If we look at an example, the test in XProcSpec looks like:

    <x:scenario label="with a required parameter, none provided.">
       <!--  <p>Requires a parameter <code>foo</code> but does not
provide any parameter. The error <code>pipx:no-parameter</code> must
be thrown.</p> -->
       <x:call step="pipx:parameter">
          <x:option name="param-name" select="'foo'"/>
          <x:option name="required" select="'true'"/>
          <!-- it appears to be a bug in xprocspec with assigning
documents to the primary parameter port, this is how it should've
worked: -->
          <!-- <x:input port="parameters"/> -->
          <!-- assigning a parameter with an unexpected name works for now: -->
          <x:param name="other" select="''"/>
       </x:call>
       <x:context label="the error document">
          <x:document type="errors"/>
       </x:context>
       <x:expect label="should contain the error 'pipx:no-parameter'"
type="xpath" test="count(/c:errors/c:error[@code='pipx:no-parameter'])
&gt; 0"/>
    </x:scenario>

and the one I wrote:

    <t:test code="param-003" step="pipx:parameter">
       <t:title>Required parameter, none provided.</t:title>
       <t:documentation>
          <p>Requires a parameter foo but does not provide any parameter.
             The error <code>pipx:no-parameter</code> must be thrown.</p>
       </t:documentation>
       <pipx:parameter param-name="foo" required="true">
          <p:input port="parameters">
             <p:empty/>
          </p:input>
       </pipx:parameter>
       <t:error code="pipx:no-parameter"/>
    </t:test>

  To be fair, if we get rid of all documentation and labels, it
becomes the following in XProcSpec:

    <x:scenario>
       <x:call step="pipx:parameter">
          <x:option name="param-name" select="'foo'"/>
          <x:option name="required" select="'true'"/>
          <x:param name="other" select="''"/>
       </x:call>
       <x:context>
          <x:document type="errors"/>
       </x:context>
       <x:expect type="xpath"
test="count(/c:errors/c:error[@code='pipx:no-parameter']) &gt; 0"/>
    </x:scenario>

and the one I wrote:

    <t:test>
       <pipx:parameter param-name="foo" required="true">
          <p:input port="parameters">
             <p:empty/>
          </p:input>
       </pipx:parameter>
       <t:error code="pipx:no-parameter"/>
    </t:test>

  I think the last one is more flexible, as it allows any sub-
pipeline to be tested (which makes the test more clear as well).
And the error assertion is definitely more clear (and I would say
error prone in XProcSpec).

  The x:context element is really bizarre.  It used to set the
context for a test of a transformation in XSpec.  Here it seems
to set the "context" for the assertions?

  I like the JUnit output, for a general purpose unit test
framework but I don't think it is useful in this case.  I like
the hierarchical report as well.

  So even though I would rather call this kind of solution an
issue, I think in this case it makes sense to keep this question
open, having both test suites set up (this is already the case),
and see what people say in the near, near, near future.  So we
can make a choice based on (little) experience.

  Regards,

-- 
Florent Georges
http://fgeorges.org/
http://h2oconsulting.be/


On 20 February 2014 19:39, Jostein Austvik Jacobsen wrote:
> - Download xprocspec (or clone its repository):
> http://daisy-consortium.github.io/xprocspec/
> - If you want validation of the xprocspec documents (useful when authoring
> them - gives autocomplete etc); add xprocspecs catalog.xml to your list of
> xml catalogs (in oxygen; edit "~/Oxygen XML Editor
> 15/frameworks/catalog.xml" and add "<nextCatalog
> catalog="/home/USERNAME/xprocspec/xprocspec/src/main/resources/META-INF/catalog.xml"/>"
> to the end.
> - To execute a xprocspec test, run
> "~/xprocspec/xprocspec/src/main/resources/content/xml/xproc/xprocspec.xpl"
> with your xprocspec test document on the primary input source port. If
> you're on Windows you should set the optional "temp-dir" option to something
> more appropriate than the default "file:/tmp/"
> - xprocspec has three secondary output ports; result, html and junit. The
> "result" port contains the raw output from the xprocspec evaluation. It's
> probably not useful to most users. The "junit" is mainly intended for maven
> integration but can be useful if you need a machine-readable format. The
> "html" port is probably what you want though; it contains a single html
> document containing the test report.
>
> I've set up a scenario in oXygen called "Basic xprocspec test" where I use
> the current document as input and point the output ports to files in
> file:/tmp/. So whenever I have a xprocspec test document open in oXygen I
> can run that scenario, then go to my browser and refresh
> "file:///tmp/xprocspec-html.html" to see the results. You could also
> instruct oXygen to automatically open the html file I suppose.
>
> I also have a oXygen scenario that lets me run the test while viewing the
> XProc document itself (to avoid switching between tabs when repeatedly
> running tests), but that's based on file nameing conventions etc. so I won't
> bother you with the details (unless you're interested).
>
> Hope this helps.
>
>
>
> wrt verbosity: I'm not sure how to make it less verbose without removing
> functionality or overloading elements with attributes. But I'm open to
> suggestions!
> wrt subpipelines: If you want to test subpipelines, then you can create a
> small wrapper script and test its inputs and outputs (which you might want
> to expose in your library anyway if it's a common pattern). By the way,
> xprocspec supports custom implementations of assetions if that helps.
>
>
> Jostein
>
>
> On 20 February 2014 19:10, Florent Georges <fgeorges@fgeorges.org> wrote:
>>
>> On 20 February 2014 18:46, Jostein Austvik Jacobsen wrote:
>>
>> > Here's an equivalent xprocspec test:
>> > https://github.com/fgeorges/pipx/pull/1/files
>>
>>   Thanks.  I accepted your pull request, I'll have a look later.  Can
>> you explain how to run it?  At first glance, I would say it is a bit
>> verbose, and it has the same drawback (someone would say advantage) as
>> XSpec has: it allows you only to execute one step, not a subpipeline.
>> What if you want to test for instance:
>>
>>     <pipx:step-a .../>
>>     <pipx:step-b .../>
>>
>> instead of just calling either pipx:step-a or pipx:step-b?
>>
>>   I will try to update http://pipx.org/progress.html soon also, but
>> for now, I have to call it a day...
>>
>>   Regards,
>>
>> --
>> Florent Georges
>> http://fgeorges.org/
>> http://h2oconsulting.be/
>
>

Received on Friday, 21 February 2014 12:54:14 UTC