Re: an idea: ports == options

Hi Jim and all,

Please see further comments/clarification inline:

> On 26 nov. 2014, at 13:29, James Fuller <jim@webcomposite.com> wrote:
> 
> in prep for todays WG a few thoughts;
> 
> Romain, I am not going to address each of your points mainly because
> the idea of conflating ports and options, whilst attractive, is highly
> improbable at this stage in xproc lifecycle.
> 
> Options provide configuration and ports define connections between
> steps ... these abstractions exist in xproc after many discussions,
> though that in itself doesn't mean we should disregard any of your
> excellent suggestions.

I perfectly understand. Thank you for taking the time to consider these ideas and discussing it in the WG :)

> 
> So if I can indulge in reductionism, I think the 'soul' of Romain's
> email hits 2 main issues;

It sounds like a correctly interpreted overview, yes.

> 
> ------------------------------------------------------------------------------
> #1) reduce xproc's verbosity in defining port connections
> ------------------------------------------------------------------------------
> 
> A side effect of Romain's proposal is some attractive simplification
> of defining port connections.
> 
> Others on the WG can correct me, but I think early on in xproc
> development it was imagined that the default implicit flow would cover
> off many common use cases and authoring explicit connections would be
> done exceptionally; experience suggests that with iterative
> development of a pipeline, the battle between the cost of rearranging
> the entire pipeline versus defining an explicit connection becomes
> pitched ... whats interesting is that once one starts defining
> explicit connection they tend to cascade as you move from an implicit
> flow to a procedural flow (on the bad days ... it kind of reminds me
> of the games I used to play with goto statements and line numbers).
> 
> Having to constantly type
> 
> <p:somestep>
> <p:input port="source">
>   <p:pipe step="someotherstep" port="result">
> </p:input>
> </p:somestep>
> 
> is a 'finger' and 'verbosity' tax that we can easily address. Actually
> I believe thats just what existing open issue #36 (and #34,#35) is
> going to address.
> 
>     https://github.com/xproc/specification/issues/36
> 
> which means we will end up with some kind of 'shorthand' attribute for
> representing p:document and p:pipe (w/ select?) at the step level
> 
> Thats a win in my eyes.

Agreed. As I mentioned in the gh issue, the “merger” part of the proposal is admittedly too ambitious for v2 (and too far off current XProc concepts).

> 
> ------------------------------------------------------------------------------
> #2) be able to access a step/port value in xpath context
> ------------------------------------------------------------------------------
> 
> Norm already did a good job explaining the difficulties of static analysis
> 
> http://lists.w3.org/Archives/Public/xproc-dev/2013Feb/0028.html
> 
> and I think that still stands.

Yes, that’s a good explanation. However note that the impossibility of doing static analysis in the case covered in Norm’s email referenced above comes from the fact that the port name is a string parameter to an XPath function, which can be dynamically computed.

In my proposal earlier in the current thread, I propose to rely instead on *static* variable binding, which doesn’t hinders static analysis IIUC.

> 
> But maybe I am missing something; one would also be to use shorthand
> (proposed in #1) with p:variable (and p:with-option, etc) which in
> turn gets exposed as an in-scope binding ( or as extern vars to xslt
> or xquery) I think we get part of the story ... right ?

It’s mostly a matter of scope. p:variable can only be declared in specific places (within a p:group, a p:choose, etc) with the scope we know.

What I propose can be seen as a shortcut mechanism to declare a p:variable scoped to a single step. E.g.

be able to do:

<px:my-step option=“fx:function($count)”>
  <p:bind name=“count” step=“earlier-step” port=“result”/>
</px:my-step>

instead of:

<p:group>
  <p:variable name=“count” select=“count(//elem)”>
    <p:pipe step=“earlier-step” port=“result” />
  </p:variable>
  <px:my-step option=“fx:function($count)” />
</p:group>




> 
> What other improvements might we consider to help the suffering xproc author ?
> 
> Personally I am doubtful of the value of exposing the default readable
> port as an implicit variable ... it seems to be redundant in that you
> already have access to this data as the effective current context;
> open to suggestions though

Yes, maybe. Depending on the changes to the current XPath context in v2. I’m thinking mostly of error XD0008, which can be a painful limitation to authors. With v1.0, it’s not easy enough to access a sequence output port in a later step’s XPath context.

> 
> Otherwise maybe some kind of implicit external variable declaration on
> xquery and xslt ???
> 
> I could imagine some kind of resource manager collecting all port
> communication and making that addressable via some kind of internal
> uri scheme ... we would not then have to make any static analysis eg.
> if it resolves it resolve, if it doesn't then we get a 404) sounds
> like an interesting custom step impl ... it may even be related to
> existing issues against logging/auditing.
> 
> So today we will spend some time going through Romain's proposal and
> seeing if we can't map parts of it to our existing issue list and if
> there are smaller level changes/amends we could take on that gets us
> part of the way there.

Thank you again for the time and effort. Keep up the good work!

Romain.

> 
> J
> 
> 
> 
> On 13 November 2014 at 12:55, James Fuller <jim@webcomposite.com> wrote:
>> tracking as
>> 
>> https://github.com/xproc/specification/issues/109
>> 
>> J
>> 
>> 
>> On 13 November 2014 12:49, James Fuller <jim@webcomposite.com> wrote:
>>> I don't feel like this got enough of an airing at the time, want to
>>> put on agenda to discuss at next WG meeting.
>>> 
>>> J
>>> 
>>> 
>>> On 16 February 2014 13:02, Romain Deltour <rdeltour@gmail.com> wrote:
>>>> Hi,
>>>> 
>>>> Just an idea for XProc v2 (or v4?):
>>>> 
>>>> TL;DR: A proposal to “merge” the concepts of ports and options (an options == a port), while still enabling static dependency graph analysis. Based on the possibility to bind readable ports to variables available in the in-scope bindings of a step.
>>>> 
>>>> Caveat: this is a pretty long read.
>>>> 
>>>> ## Rationale
>>>> 
>>>> XProc v2 will allow arbitrary XDM values in variables and options [1]. This blurs the line between options and ports, both will accept sequences of documents, or as likely in v2, any XDM. The primary difference is that ports can be *connected*.
>>>> But, if there was a way to consume a step result from the XPath representing an option’s value, that would essentially mean that output ports can be connected to options. In other words, it kind of erases all the conceptual differences between ports and options.
>>>> 
>>>> About a year ago, it was suggested to use a *function* (say ‘p:read-port') to access ports from XPath expressions [2]. This idea was not accepted as a v2 req based on the very legit grounds that the static analysis of the XProc graph would become impractical [3].
>>>> 
>>>> What I’m considering below is to use another mechanism to make readable ports available to XPath expressions, not using a function but rather externally binding readable ports to variables.
>>>> 
>>>> ## Proposal
>>>> 
>>>> 1. First, assume that there are no longer differences between input ports and options. An input port is an option, an option is an input port.
>>>> 
>>>> 2. Introduce a declarative way to bind a readable port to a variable, added to the in-scope bindings of the Environment. This could be done with a new element, say “p:bind”, with a required “name” attribute (the name of the variable), and either a “step” and “port” pair of attributes (to bind the variable to a readable port) or inline content.
>>>> 
>>>> 3. The options / ports are set by XPath expressions, the previously defined variables being available from the static XPath context.
>>>> 
>>>> ## Connections
>>>> 
>>>> (adapted from the examples in the XProc spec “Associating Documents with Ports" [4])
>>>> 
>>>> ### Specified by URI
>>>> 
>>>> Would be done with fn:doc.
>>>> 
>>>> <p:identity source="doc('http://example.com/input.xml')"/>
>>>> 
>>>> ### Specified by Source
>>>> 
>>>> Declaratively bind the source to a variable with the p:bind element, then use this binding in the option declaration (see more of the nitty-gritty in the later section below).
>>>> 
>>>>  <p:xinclude source=“$source”>
>>>>    <p:bind name=“source” step=“other” port=“results”/>
>>>>  <p:xinclude>
>>>> 
>>>> ### Specified inline
>>>> 
>>>> Use the p:bind element with inline content.
>>>> 
>>>>  <p:xslt stylesheet=“$stylesheet">
>>>>    <p:bind name=“stylesheet”>
>>>>      <xsl:stylesheet version=“1.0”>…</xsl:stylesheet>
>>>>    </p.bind>
>>>>  </p:xslt>
>>>> 
>>>> ### Specified explicitly empty
>>>> 
>>>> Use an empty attribute, or the empty sequence.
>>>> 
>>>>  <p:xslt source=“” stylesheet=“doc(‘stylesheet.xsl')”/>
>>>> 
>>>> ## Nitty gritty of the "port” connections
>>>> 
>>>> ### Implicit connections
>>>> 
>>>> An important and convenient feature of port connections in v1 is the concept of primary ports and default connections. Similarly, one option in the signature could be annotated as the “primary” option, which would be automatically bound to the default readable port. That would keep looking like that:
>>>> 
>>>> <p:identity/>
>>>> 
>>>> being equivalent to
>>>> 
>>>>  <p:identity source=“$source">
>>>>    <p:bind name=“source” step=“previous-step” port=“primary-output”/>
>>>>  </p:identity/>
>>>> 
>>>> Another possibility is to reserve the $default variable to automatically bind it to the default readable port:
>>>> 
>>>>  <p:identity source=“$default”/>
>>>> 
>>>> ### Variable overrides
>>>> 
>>>> You’ll asks: What about existing variables from the in-scope bindings ? Is there a risk to override them ?
>>>> Well, the idea is that yes, p:bind overrides previous variables.
>>>> 
>>>> For instance in:
>>>> 
>>>>  <p:variable name=“source” value=“‘blahblah’/>
>>>>  <p:identity source=“$source”>
>>>>    <p:bind name=“source” step=“previous” port=“result”/>
>>>>  <p:identity/>
>>>> 
>>>> the source option would be bound to the result of the previous step, the variable source is overridden in the in-scope bindings.
>>>> If you want to keep using that variable, there’s still the option of using another name for the port binding:
>>>> 
>>>>  <p:variable name=“source” value=“‘blahblah’/>
>>>>  <p:identity source=“$my-source”>
>>>>    <p:bind name=“my-source” step=“previous” port=“result”/>
>>>>  <p:identity/>
>>>> 
>>>> ### Implicit options
>>>> 
>>>> It might be useful to implicitly set options if there is a variable of the same name in the in-scope bindings. For instance:
>>>> 
>>>>  <p:identity> <!-- the source option is implicitly set -->
>>>>    <p:bind name=“source” step=“previous” port=“result”/>
>>>>  <p:identity/>
>>>> 
>>>> ## More complex examples
>>>> 
>>>> Logging the count of flowing documents using a potential p:message step of v2:
>>>> 
>>>>  <p:message source=“$source” message="'number of docs: ' || count($source)”>
>>>>    <p:bind name="source" step="other" port="result"/>
>>>>  </p:message>
>>>> 
>>>> or, possibly with a reserved $default variable and implicit connections:
>>>> 
>>>>  <p:message message="'number of docs: ' || count($default)”/>
>>>> 
>>>> (note: $default is bound to the default readable port by convention, @source is declared in the signature as the primary option so it’s implicitly connected to the default readable port).
>>>> 
>>>> Try that with XProc v1 :)
>>>> 
>>>> That’s it! I’ve probably overlooked a bazillion things, but just wanted to throw the idea while it’s fresh. Comments welcome!
>>>> 
>>>> Romain.
>>>> 
>>>> [1] http://www.w3.org/XML/XProc/docs/xproc-v2-req.html#aribrary-vars
>>>> [2] http://lists.w3.org/Archives/Public/xproc-dev/2013Feb/0028.html
>>>> [3] http://lists.w3.org/Archives/Public/public-xml-processing-model-comments/2013Sep/0001.html
>>>> [4] http://www.w3.org/TR/xproc/#syntax-docs-ports

Received on Wednesday, 26 November 2014 14:26:04 UTC