Re: “Packaging”

It was interesting for me to see that we all seem to focus on different packaging aspects. Maybe we’ll first need to find out what our common main goals are:

1. Do we only want to facilitate support for features that can be implemented in pure XPath (and exclude things like e.g. a possible fn:unparsed-binary function, an SFTP, SQL, Mail or File module, etc… Things that EXPath Packaging allows us to do)?

2. Would we like to have a static approach (a preprocessing step to retrieve libraries/modules, as known from maven or npm) or something dynamic (resolve unknown dependencies at parse time)?

3. Are we rather looking for a mechanism that creates XQuery and XSLT syntax from an XML representation, or

4. Are we aiming for a platform that allows arbitrary users to contribute new functionality?



Am 10.11.2023 23:14 schrieb Sasha Firsov <suns@firsov.net>:
Apologies upfront of swinging the conversation.
XSLT/XPath needs to enable the capabilities of libraries/packaging in at least two aspects.
1. The package definition. Similar to package.json in NPM. To keep as source as binary(-es) referenced. As esm vs csm can be defined in package.json, the target standard and execution environment have to be listed with their own dependencies. There is a lot to improve there.

2. Module mapping. For reference the dependencies via symbolic name. The mapping to actual implementation would use the package definition and can be overridden in each layer. That would enable the abstraction necessary for swapping and inherent implementations.
-s

On Fri, Nov 10, 2023, 09:13 Christian Grün <cg@basex.org<mailto:cg@basex.org>> wrote:
Hi Norm,

I think I agree with all your observations: It feels essential to have a unified packaging mechanism; both implementors, contributors and developers would benefit a lot from it. FunctX I believe is a great example for that, although it’s “just a library” and limited in is scope, but it has been used a lot with different processors (as far as I can judge). The EXPath Packaging was at least an honorable attempt. I felt the main deficiency was that every implementation used/embedded it differently. Next, I remember the interesting qtspecs discussion on that topic [1].

I also agree it feels out of scope to spend really much time on it in our group, so I definitely appreciate the intent to achieve a lot with little effort :)

Again it's the XQuery prospective I advocate, and the easiest approach that I can imagine would be to:

• have a central repository for downloading modules with a given path and
• address these modules with the existing XQuery “import module namespace 'xyz';” statement.
• If an implementation exists for the URI in the processor, or if the module has already been downloaded, the repository access can be skipped.

That's it. Basically. But of course some challenges start just here:

• Do we only want to support XQuery library modules?
• If we support other languages (such as Java/JavaScript code, or any other language than can be run by the invoking processors), how do we define entry points (function signatures)? The approach we chose for “Java Modules” would probably not be generic enough [2].
• How can specific versions of the module be addressed? Possibly with an optional ‘version' query parameter in the module URI?
• Or would it be better to define dependencies outside the code (see Maven and similar)?

And I have no idea how that would match with XSLT…

My two cents,
Christian

[1] https://github.com/qt4cg/qtspecs/issues/274

[2] https://docs.basex.org/wiki/Repository#Java




Am 10.11.2023 12:13 schrieb Norm Tovey-Walsh <norm@saxonica.com<mailto:norm@saxonica.com>>:
Hello,

At Declarative Amsterdam this year, Adam Retter gave a compelling talk
(EXPath Pkg Version 2) about the possibility of doing some sort of
packaging work.

We’ve talked about this a few times, in at least a couple of different
contexts. I think we’d all agree that Maven, NuGet, NPM, pypi, etc. are
a significant part of their respective communities.

It would be good if we had something like that too. It’s a very large
undertaking and solving that problem is absolutely out-of-scope for us.

However…

Could we spec out the bare minimum needed to support some sort of
packaging for XQuery and XSLT? (This is related to, but probably
distinct from xsl:package packages; the fact that there’s going to be
overlap there is just an unfortunate reality.)

I’m motivated to bring this up now both because it would be better to
have the hooks in place if/when some future packaging standard takes
off, and because we might allow vendors to leverage the mechanism in
non-standard ways in the short term.

I’m also thinking about all of the functions we’re seeing proposed. I
fear the list of useful functions we could standardize is endless and
we’ve already reached a point where it would be challenging to
articulate a principled account of when we do or do not standardize
fn:usefulthing.

My naïve, I’ve spent a whole 10 minutes thinking about this, story goes
something like this:

Suppose a query could begin with

   some-syntax-for-packages "array-extensions"

and that would make a family of array extension functions available
(perhaps even in the fn: namespace) or

  <xsl:some-syntax-for-packages module="cryptography"/>

and that would make a family of cryptographic functions available.

Vendors could roll their own with the expectation that when an
interoperable packaging standard is available, they’ll support that too.

I dunno. But it feels like we’ll regret it if we don’t make the hooks
available.

                                        Be seeing you,
                                          norm

--
Norm Tovey-Walsh
Saxonica