generators (was:) Re: QT4CG meeting 148 draft agenda, 13 January 2026

On Mon, 12 Jan 2026 11:21:03 +0000
Norm Tovey-Walsh <norm@saxonica.com> wrote:

> 
> 2.1. PR #2350: 708 An alternative proposal for generators
> 
>    See PR [37]#2350.
> 

Thanks, i’ll try to be there, and i also enclose the beginnings of some
use cases (in case anyone has time to read them, sorry for short notice
here)

For me, it’s not that there is anything that can’t be done without
generators, since Dimitre has implemented them in qt4 for BaseX
already. And it might be that a much smaller, core, proposal is enough.

You can also implement regular expressions yourself in XPath, as an
extreme example, using codepoint and string manipulation, but you
seriously don’t want to.

Here, generators are small, and are a common paradigm. I don’t want to
overstate the case for them - an alternative might be to consider an
equivalent to xsl:iterate for XPath (and hence XQuery): a function form
that’s guaranteed to be optimized into a loop. But that’s lower level.

If we had a standard paradigm for writing generators, they would be
used for the random number generator generator. Its exactly the same
idea: you call random-number-generator($seed?) and you get back a
generator. You then use that generator repeatedly to get random numbers.

Generators are useful to abstract that idea; some examples:

1. Mutual Recursion

Mutual recursion is common e.g. in recursive descent parsers, where
multiple functions need to operate on the next token in the input,
possibly hiding things like macro expansion (e.g. parameter entites)
from the parser. So you make a get-next-token generator whose next()
function can go off into another file when needed.

2. Avoiding computation

Example: Take two documents with roughly the same content but not in
the same order, and use text similarity to match paragraphs.

Here, the similarity function is expensive to compute, but a simple
approximation is enough to classify paragraphs into obviously
dissimilar and possibly similar. So you want a next-most-likely
function, but you don't want to compute similarity for the whole
sequence in advance.

3. N-way merge

Example: make a sequence of up to _k_ sections from multiple documents
based on section title or keyword relevance; each document can contain
multiple sections, and each section must be included only once, based
on the topic sequence in the original.

The usual way to do this in “QT” is with a recursive function or
template that chooses a single element at each level, but having a
generator that can return several elements when appropriate can
considerably simplify the logic.

Of course, you can write this with helper functions (and i do, today).
None of this is about things you can’t do today, but only about things
that are tricky to get right, and bringing in familiar ideas from other
languages.

A fold-left-while, by analogy with take-while, might be a useful
addition for solving complex problems, but that goes back to
xsl:iterate too.

hope this helps,

liam

-- 
Liam Quin: Delightful Computing - Training and Consultancy in
XSLT / XML Markup / Typography / CSS / Accessibility / and more...
Outreach for the GNU Image Manipulation Program
Vintage art digital files - fromoldbooks.org

Received on Tuesday, 13 January 2026 01:10:59 UTC