Need to add SPC

Hello

First of all thanks to all of those who spent so much time and effort to get
the details of this concept hammered out. I think it looks great!
Special thanks to Michael Sperberg-McQueen who hung on to the concept from
1993 to 2000 and promoted it in the W3C with the result that we now have
Xproc. Kudos to Michael.

I do have two items - one a nit, the other more serious.

The nit

I still prefer the name 'assembly line' instead of pipeline'.
Reasons:
A pipeline transports while an assembly line manufactures or creates.
Xproc is an assembly line for manufacturing or creating information not a
transportation mechanism.

This is a nit since 'pipeline' seems to be set in stone already.
But it does point out maybe why the following more serious issue was
mislaid.

SPC
XProc did a good job of providing the 'attachment' and the 'association'
principles for the creation of assembly lines mentioned in 1993.

Xproc error capabilities identify if the step does not conform to the XProc
specification and provides a mechanism for the step internals to report
using its own built-in error capabilities. Basic programming viewpoint.

However, XProc does not include the use of Statistical Process Control (SPC)
to maintain and improve quality of product produced by the step internals.
This capability is essential for ....lines to maintain quality and should be
part of the line not dependent on the developers of steps to include.

An example from the industrial age:
Assume an assembly line which processes metal stock with a lathe to create
widgets.
The supplier of metal stock changed its process or we changed suppliers and
the input metal stock is now a softer metal.
The lathe blithely continues to create widgets but because of the softer
metal stock the widgets end up being 1/100 inch smaller which downstream
results in widgets that are too loose when fit into the widget holder.
Customer complaints increase. Going back we find that the last 100,000
widgets are too small.
If SPC was in place and every 100th widget was routed to quality control for
testing, this problem would have been found and fixed long before the surge
in customer complaints or 100,000 widgets had been wasted.

We need a similar capability for the information age. When dealing with
text, natural language and so on, we have to recognize that the inputs may
change. The power of language comes from its ability to create new concepts
by the derivation of previous words, adding semantics to old words, and
creation of new words. We need SPC capabilities in order to maintain the
quality of the ...lines we create.

Other examples of value:
......lines allow us to replace a machine with a new and hopefully better
machine.
How do we verify that the new machine does what its supposed to do?
Again if we had SPC we could plug in the machine into the assembly line and
verify the results.

Say we have a line consisting of 100 steps.  We know that the end result is
only 80% correct.  How do we find the culprit in the pipeline that is
causing most of the 20% failure rate.  With SPC we can test various steps to
identify which step to focus our energies on.

So I suggest:

SPC capabilities to be included in each step:
SPC can be turned on or off. When 'off' no output is sent to quality
control. When 'off' the pass through test should be as efficient as possible
so that SPC does not side effect processing speed.
When 'on' every 'xth' output is routed to quality control.

This can be done with a simple integer - 0 = off, positive integer indicates
how often a result is pulled off the line and checked for quality. A value
of 10 indicates every 10th result is routed to quality control. A value of
18 indicates every 18th results is routed to quality control.

Can be dynamically changed.

An option that may be considered:  Copy is routed, original continues on
through the line.

Again thank you for your time and effort to make this a reality.

Have a cup of tea
=========================

Lloyd Harding
218-962-3202