1 Introduction
A large and growing set of specifications describe processes operating on XML
documents. Many applications depend on the use of more than one of the many inter-related
XML family of specifications. How implementations of these specifications interact affects
interoperability. XProc: An XML Pipeline
Language is designed for describing operations to be performed on XML
documents.
This specification contains requirements for an anticipated XProc
V.next. This specification is concerned with the conceptual model of XML process
interactions, the language for the description of these interactions, and the inputs and
outputs of the overall process. This specification is not generally concerned with the
implementations of actual XML processes participating in these interactions.
1.1 XProc V.next Goals
| Editorial note | |
| The editors intend to enumerate the Working Group's goals for XProc
V.next to guide our efforts, and these may ultimately inform 3 Design Principles. |
The following is a strawman list; it has no standing with the Working Group and is likely
to be replaced and/or expanded daily until further notice.
Review Definitions. Section 2.
Review Design Principles. Section 3.
Gather and review Related Specifications. Appendix A.
Gather and review outstanding issues. Appendix B.
Audit existing requirements and use cases. Appendix C
Gather and review collected input. Appendix D
Gather and review categorized list of extant steps. Appendix E
Update existing definitions, design principles, requirements and use cases.
Gather and review input from stakeholders under the following categories. See
Appendix D.
Architecture
Resource Management.
Integration.
Usability.
New Steps
Discuss.
Enumerate new design principles, requirements and use cases.
Publish.
4 Requirements
4.1 Standard Names for Component Inventory
The XML Pipeline Specification Document must have
standard names for components that correspond, but not limited to, the following
specifications [xml-core-wg]:
| Editorial note: Satisfied | 20120407 |
| This requirement is satisfied by XProc 1.0. Refer to D Pipeline Language
Summary. The following are code snippets which exemplify related XProc 1.0
step names. |
<p:xinclude ... />
<p:xslt name="make-fo" ... >
<p:xsl-formatter ... />
<p:validate-with-xml-schema name="validated" ... >
<p:validate-with-relax-ng ... />
<p:xquery ... />
| Editorial note: Supported | 20120407 |
| Editors have added the following list to reflect normatively referenced
specifications in XProc 1.0. |
The following list is proposed as added requirements on the basis that they
have been accomplished in XProc 1.0, as reflected in the related example. Moreover, these
specifications are cited normatively in XProc 1.0.
<p:namespaces ... />
<p:namespace-rename .... />
<p:validate-with-schematron ... />
<p:xpath-context .... />
<p:xpath-version .... />
<p:xpath-version-available ... />
<c:http-request" ... />
The following list is proposed as added requirements on the basis these
specifications have been stipulated by reference in XProc 1.0.
4.2 Allow Defining New Components and Steps
An XML Pipeline must allow applications to define
and share new steps that use new or existing components. [xml-core-wg]
| Editorial note: Satisfied | 20120407 |
| Defining and sharing steps is satisfied by XProc 1.0; see the Introduction. The
ability to define additional step types is Implementation-defined. See the list of reference implementations for
examples of components and steps that are defined ex-cathedra. |
4.3 Minimal Component Support for Interoperability
There must be a minimal inventory of components
defined by the specification that are required to be supported to facilitate
interoperability of XML Pipelines.
| Editorial note: Minimal is Moot | 20120407 |
| Failing a definition of "minimal inventory" this requirement cannot be satisfied.
Is this requiring a minimal number of components to satisfy interoperability among only
those components listed in 4.1? In that case, we need to work on 4.1. Otherwise, the
editors consider this requirement to be self-evidently moot; "minimal inventory" cannot
be defined or measured to the satisfaction of any three people. |
4.4 Allow Pipeline Composition
Mechanisms for XML Pipeline composition for re-use
or re-purposing must be provided within the XML Pipeline
Specification Document.
| Editorial note: Satisfied | 20120407 |
| This requirement is satisfied by XProc 1.0, Example 1. There may be room for
improvement. reflected in the Collected Input. |
4.5 Iteration of Documents and Elements
XML Pipelines should allow iteration of a specific
set of steps over a collection of documents and or elements
within a document.
| Editorial note: Satisfied | 20120407 |
| This requirement is satisfied by XProc 1.0, Example 3. The term "specific set" was
never defined. We should probably do that now, in the corrigendum. We have opportunity
to provide advice here, again and again. |
4.6 Conditional Processing of Inputs
To allow run-time selection of steps, XML Pipelines should provide mechanisms for conditional
processing of documents or elements within documents based on expression evaluation.
[xml-core-wg]
| Editorial note: Satisfied | 20120407 |
| This requirement is satisfied by XProc 1.0, Example 3. What principles have we
learned about conditional processing of inputs? |
4.7 Error Handling and Fall-back
XML Pipelines must provide mechanisms for
addressing error handling and fall-back behaviors. [xml-core-wg]
| Editorial note: Satisfied | 20120407 |
| This requirement is satisfied by XProc: All steps have an implicit output port for
reporting errors; error handling and fallback are manageable through use of p:try. and p:catch. |
<p:try name? = NCName> (p:variable*, p:group, p:catch) </p:try>
4.8 Support for the XPath 2.0 Data Model
XML Pipelines must support the XPath 2.0 Data Model
to allow support for XPath 2.0, XSLT 2.0, and XQuery as steps.
Note:
At this point, there is no consensus in the working group that minimal conforming
implementations are required to support the XPath 2.0 Data Model.
| Editorial note: Satisfied | 20120407 |
| This requirement is satisfied in XProc 1.0, with the caveats noted in 2.6 XPaths in XProc.
|
4.9 Allow Optimization
An XML Pipeline should not inhibit a sophisticated
implementation from performing parallel operations, lazy or greedy processing, and other
optimizations. [xml-core-wg]
4.10 Streaming XML Pipelines
An XML Pipeline should allow for the existence of
streaming pipelines in certain instances as an optional optimization. [xml-core-wg]
| Editorial note: Satisfied | 20120407 |
| This requirement is more than satisfied by XProc 1.0, except as noted for
7.1.23 p:split-sequence |
5 Use cases
This section contains a set of use cases that supported our requirements and informed our
design. While there was a want to address all the use cases listed in this document, in the
end, the first version may not have solved all the following use cases. Those unsolved use
cases may be migrated into XProc V.next.
5.1 Apply a Sequence of Operations
Apply a sequence of operations such XInclude, validation, and transformation to a
document, aborting if the result or an intermediate stage is not valid.
(source: [xml-core-wg])
| Editorial note: Satisfied | 20120407 |
| This use case is satisfied by Examples 1-3 in the Introduction |
5.2 XInclude Processing
Retrieve a document containing XInclude instructions.
Locate documents to be included.
Perform XInclude inclusion.
Return a single XML document.
(source: Erik
Bruchez)
| Editorial note: Satisfied | 20120407 |
| This use case is satisfied by Examples 1-3 in the Introduction |
5.3 Parse/Validate/Transform
Parse the XML.
Perform XInclude.
Validate with Relax NG, possibly aborting if not valid.
Validate with W3C XML Schema, possibly aborting if not valid.
Transform.
(source: Norm
Walsh)
| Editorial note: Satisfied | 20120407 |
| This use case is almost satisfied by Examples 1-3 in the Introduction. The example
does not include Relax NG validation, but it could have, and Schematron as well.
|
5.4 Document Aggregation
Locate a collection of documents to aggregate.
Perform aggregation under a new document element.
Return a single XML document.
(source: Erik
Bruchez)
| Editorial note: Satisfied | 20120407 |
| This use case is satisfied by XProc 1.0 as exemplified in p:for-each |
5.5 Single-file Command-line Document Processing
Read a DocBook document.
Validate the document.
Process it with XSLT.
Validate the resulting XHTML.
Save the HTML file using HTML serialization.
(source: Erik
Bruchez)
| Editorial note: Satisfied | 20120407 |
| Although the processing scenario described above is exemplified in p:for-each, the command-line
requirement is considered to be implementation defined. ["How outside values are
specified for pipeline parameters on the pipeline initially invoked by the processor is
implementation-defined. In other words, the command line options, APIs, or other
mechanisms available to specify such parameter values are outside the scope of [XProc
1.0]."] |
5.6 Multiple-file Command-line Document Generation
Read a list of source documents.
For each document in the list:
Read the document.
Perform a series of XSLT transformations.
Serialize each result.
Alternatively, aggregate the resulting documents and serialize a single result.
(source: Erik
Bruchez)
| Editorial note: Satisfied | 20120407 |
| Although the processing scenario described above is exemplified in p:for-each, the command-line
requirement is considered to be implementation defined. ["How outside values are
specified for pipeline parameters on the pipeline initially invoked by the processor is
implementation-defined. In other words, the command line options, APIs, or other
mechanisms available to specify such parameter values are outside the scope of [XProc
1.0]."] |
5.7 Extracting MathML
Extract MathML fragments from an XHTML document and render them as images. Employ an SVG
renderer for SVG glyphs embedded in the MathML.
(source: [xml-core-wg])
| Editorial note: MathML Unsatisfied | 20120407 |
| This use case is not satisfied. |
5.8 Style an XML Document in a Browser
Style an XML document in a browser with one of several different stylesheets without
having multiple copies of the document containing different xml-stylesheet directives.
(source: [xml-core-wg])
| Editorial note: Satisfied | 20120407 |
| This use case is satisfied by XProc 1.0 as exemplified in p:for-each |
5.9 Run a Custom Program
Run a program of your own, with some parameters, on an XML file and display the result in
a browser.
(source: [xml-core-wg])
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.10 XInclude and Sign
Process an XML document through XInclude.
Transform the result with XSLT using a fixed transformation.
Digitally sign the result with XML Signatures.
(source: Henry
Thompson)
| Editorial note: XML Signatures Unsatisfied | |
| This use case is not satisfied. |
5.11 Make Absolute URLs
Process an XML document through XInclude.
Remove any xml:base attributes anywhere in the resulting document.
Schema validate the document with a fixed schema.
For all elements or attributes whose type is xs:anyURI, resolve the value against the
base URI to create an absolute URI. Replace the value in the document with the
resulting absolute URI.
This example assumes preservation of infoset ([base URI]) and PSVI ([type definition])
properties from step to step. Also, there is no way to reorder these steps as the schema
doesn't accept xml:base attributes but the expansion requires xs:anyURI typed values.
(source: Henry
Thompson)
| Editorial note: Satisfied | 20120407 |
| This use case is satisfied by XProc 1.0 as exemplified in 7.2.10 p:xsl-formatter |
5.12 A Simple Transformation Service
Extract XML document (XForms instance) from an HTTP request body
Execute XSLT transformation on that document.
Call a persistence service with resulting document
Return the XML document from persistence service (new XForms instance) as the HTTP
response body.
(source: Erik
Bruchez)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.13 Service Request/Response Handling on a Handheld
Allow an application on a handheld device to construct a pipeline, send the pipeline and
some data to the server, allow the server to process the pipeline and send the result
back.
(source: [xml-core-wg])
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.14 Interact with Web Service (Tide Information)
Parse the incoming XML request.
Construct a URL to a REST-style web service at the NOAA (see website).
Parse the resulting invalid HTML document with by translating and fixing the HTML to
make it XHTML (e.g. use TagSoup or tidy).
Extract the tide information from a plain-text table of data from document by
applying a regular expression and creating markup from the matches.
Use XQuery to select the high and low tides.
Formulate an XML response from that tide information.
(source: Alex
Milowski)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.15 Parse and/or Serialize RSS descriptions
Parse descriptions:
Iterate over the RSS description elements and do the following:
Gather the text children of the 'description' element.
Parse the contents with a simulated document element in the XHTML namespace.
Send the resulting children as the children of the 'description element.
Apply rest of pipeline steps.
Serialize descriptions
Iterate over the RSS description elements and do the following:
Serialize the children elements.
Generate a new child as a text children containing the contents (escaped
text).
Apply rest of pipeline steps.
(source: Alex
Milowski)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.16 XQuery and XSLT 2.0 Collections
In XQuery and XSLT 2.0 there is the idea of an input and output collection and a pipeline
must be able to consume or produce collections of documents both as inputs or outputs of
steps as well as whole pipelines.
For example, for input collections:
Accept a collection of documents.
Apply a single XSLT 2.0 transformation that processes the collection and produces
another collection.
Serialize the collection to files or URIs.
For example, for output collections:
Accept a single document as input.
Apply an XQuery that produces a sequence of documents (a collection).
Serialize the collection to files or URIs.
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.17 An AJAX Server
Receive XML request with word to complete.
Call a sub-pipeline that retrieves list of completions for that word.
Format resulting document with XSLT.
Serialize response to XML.
(source: Erik
Bruchez)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.18 Dynamic XQuery
Dynamically create an XQuery query using XSLT, based on input XML document.
Execute the XQuery against a database.
Construct an XHTML result page using XSLT from the result of the query.
Serialize response to HTML.
(source: Erik
Bruchez)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.19 Read/Write Non-XML File
Read a CSV file and convert it to XML.
Process the document with XSLT.
Convert the result to a CSV format using text serialization.
(source: Erik
Bruchez)
| Editorial note: TBD | 20120407 |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.20 Update/Insert Document in Database
Receive an XML document to save.
Check the database to see if the document exists.
If the document exists, update the document.
If the document does not exists, add the document.
(source: Erik
Bruchez)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.21 Content-Dependent Transformations
Receive an XML document to format.
If the document is XHTML, apply a theme via XSLT and serialize as HTML.
If the document is XSL-FO, apply an XSL FO processor to produce PDF.
Otherwise, serialize the document as XML.
(source: Erik
Bruchez)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.22 Configuration-Dependent Transformations
Mobile example:
Receive an XML document to format.
If the configuration is "desktop browser", apply desktop XSLT and serialize as
HTML.
If the configuration is "mobile browser", apply mobile XSLT and serialize as
XHTML.
News feed example:
Receive an XML document in Atom format.
If the configuration is "RSS 1.0", apply "Atom to RSS 1.0" XSLT.
If the configuration is "RSS 2.0", apply "Atom to RSS 2.0" XSLT.
Serialize the document as XML.
(source: Erik
Bruchez)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.23 Response to XML-RPC Request
Receive an XML-RPC request.
Validate the XML-RPC request with a RelaxNG schema.
Dispatch to different sub-pipelines depending on the content of
/methodCall/methodName.
Format the sub-pipeline response to XML-RPC format via XSLT.
Validate the XML-RPC response with an W3C XML Schema.
Return the XML-RPC response.
(source: Erik
Bruchez)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.24 Database Import/Ingestion
Import example:
Read a list of source documents.
For each document in the list:
Validate the document.
Call a sub-pipeline to insert content into a relational or XML database.
Ingestion example:
Receive a directory name.
Produce a list of files in the directory as an XML document.
For each element representing a file:
Create an iTQL query using XSLT.
Query the repository to check if the file has been uploaded.
Upload if necessary.
Inspect the file to check the metadata type.
Transform the document with XSLT.
Make a SOAP call to ingest the document.
(source: Erik
Bruchez)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.25 Metadata Retrieval
Call a SOAP service with metadata format as a parameter.
Create an iTQL query with XSLT.
Query a repository for the XML document.
Load a list of XSLT transformations from a configuration.
Iteratively execute the XSLT transformations.
Serialize the result to XML.
(source: Erik
Bruchez)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.26 Non-XML Document Production
A non-XML document is fed into the process.
That input is converted into a well-formed XML document.
A table of contents is extracted.
Pagination is performed.
Each page is transformed into some output language.
(source: Rui
Lopes)
Read a non-XML document.
Transform.
(source: Norm
Walsh)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.27 Integrate Computation Components (MathML)
Select a MathML content element.
For that element, apply a computation (e.g. compute the kernel of a matrix).
Replace the input MathML with the output of the computation.
(source: Alex
Milowski)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.28 Document Schema Definition Languages (DSDL) - Part 10: Validation Management
This document provides a test scenario that will be used to create validation management
scripts using a range of existing techniques, including those used for program
compilation, etc.
The steps required to validate our sample document are:
Use ISO 19757-4 Namespace-based Validation Dispatching Language (NVDL) to split out
the parts of the document that are encoded using HTML, SVG and MathML from the bulk of
the document, whose tags are defined using a user-defined set of markup tags.
Validate the HTML elements and attributes using the HTML 4.0 DTD (W3C XML DTD).
Use a set of Schematron rules stored in check-metadata.xml to ensure that the
metadata of the HTML elements defined using Dublin Core semantics conform to the
information in the document about the document's title and subtitle, author, encoding
type, etc.
Validate the SVG components of the file using the standard W3C schema provided in the
SVG 1.2 specification.
Use the Schematron rules defined in SVG-subset.xml to ensure that the SVG file only
uses those features of SVG that are valid for the particular SVG viewer available to
the system.
Validate the MathML components using the latest version of the MathML schema (defined
in RELAX-NG) to ensure that all maths fragments are valid. The schema will make use
the datatype definitions in check-maths.xml to validate the contents of specific
elements.
Use MathML-SVG.xslt to transform the MathML segments to displayable SVG and replace
each MathML fragment with its SVG equivalent.
Use the ISO 19757-8 Document Schema Renaming Language (DSRL) definitions in
convert-mynames.xml to convert the tags in the local nameset to the form that can be
used to validate the remaining part of the document using docbook.dtd.
Use the IS0 19757-7 Character Repertoire Definition Language (CRDL) rules defined in
mycharacter-checks.xml to validate that the correct character sets have been used for
text identified as being Greek and Cyrillic.
Convert the Docbook tags to HTML so that they can be displayed in a web browser using
the docbook-html.xslt transformation rules.
Each validation script should allow the four streams produced by step 1 to be run in
parallel without requiring the other validations to be carried out if there is an error in
another stream. This means that steps 2 and 3 should be carried out in parallel to steps 4
and 5, and/or steps 6 and 7 and/or steps 8 and 9. After completion of step 10 the HTML
(both streams), and SVG (both streams) should be recombined to produce a single stream
that can fed to a web browser. The flow is illustrated in the following diagram:
(source: Martin
Bryan)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.29 Large-Document Subtree Iteration
Running XSLT on a very large document isn't typically practical. In these cases, it is
often the case that a particular element, that may be repeated over-and-over again, needs
to be transformed. Conceptually, a pipeline could limit the transformation to a subtree
by:
Limiting the transform to a subtree of the document identified by an XPath.
For each subtree, cache the subtree and build a whole document with the identified
element as the document element and then run a transform to replace that subtree in
the original document.
For any non-matches, the document remains the same and "streams" around the
transform.
This allows the transform and the tree building to be limited to a small subtree and the
rest of the process to stream. As such, an arbitrarily large document can be processed in
a bounded amount of memory.
(source: Alex
Milowski)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.30 Adding Navigation to an Arbitrarily Large Document
For a particular website, every XHTML document needs to have navigation elements added to
the document. The navigation is static text that surrounds the body of the document. This
navigation is added by:
Matching the head and body elements using a XPath expression that can be
streamed.
Inserting a stub for a transformation for including the style and surrounding
navigation of the site.
For each of the stubs, transformations insert the markup using a subtree expansion
that allows the rest of the document to stream.
In the end, the pipeline allows arbitrarily large XHTML document to be processed with a
near-constant cost.
(source: Alex Milowski)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.31 Fallback to Choice of XSLT Processor
A step in a pipeline produces multiple output documents. In XSLT 2.0, this is a standard
feature of all XSLT 2.0 processors. In XSLT 1.0, this is not standard.
A pipeline author wants to write a pipeline that, at compile-time, the implementation
chooses XSLT 2.0 when possible and degrades to XSLT 1.0 when XSLT 2.0 is not supported. In
the case of XSLT 1.0, the step will use XSLT extensions to support the multiple output
documents--which again may fail. Fortunately, the XSLT 1.0 transformation can be written
to test for this.
(source: Alex Milowski)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |
5.32 No Fallback for XQuery Causes Error
As the final step in a pipeline, XQuery is required to be run. If the XQuery step is not
available, the compilation of the pipeline needs to fail. Here the pipeline author has
chosen that the pipeline must not run if XQuery is not available.
(source: Alex Milowski)
| Editorial note: TBD | |
| This use case is [not] satisfied by XProc 1.0 as exemplified in TBD |