[ANN] xqDoc XQuery code documentation generator version 1.9.3 is released

Announcing an updated xqDoc (http://xqdoc.org <http://xqdoc.org/>) XQuery code documentation generator!

Thank you to Darin McBeath and Curt Kohler from Elsevier that originally developed xqDoc.

The previous version had not been updated to deal with the recent additions to the XQuery grammar.  It was also developed using Antlr 2 and Antlr 4 is the current release.  Antlr 4 allows for the separation of the parsing of the grammar and the visiting of the parsed grammar.  This allows for the grammar file to be utilized for other parsing reasons by writing a new visitor class.

The source is available at:  https://github.com/lcahlander/xqdoc <https://github.com/lcahlander/xqdoc> and the jar file is available through the maven repository https://mvnrepository.com/artifact/org.xqdoc/xqdoc <https://mvnrepository.com/artifact/org.xqdoc/xqdoc>

1.0 Additions

The parsing of the content of a function signature was in the specification, but not in the available codebase.
The following XQuery function now generates parameter, return type and annotation tags within the function xqDoc tag.

1.1 Function Signature

declare
    %rest:GET
    %rest:path("/test2")
    %rest:query-param("hello", "{$hello}", "")
function test:hello2($hello as xs:string*)
as node()
{
    <hello>{$hello}</hello>
};


1.1.1 Parameters

            <xqdoc:parameters>
                <xqdoc:parameter>
                    <xqdoc:name>hello</xqdoc:name>
                    <xqdoc:type occurrence="*">xs:string</xqdoc:type>
                </xqdoc:parameter>
            </xqdoc:parameters>

1.1.2 Return Types


            <xqdoc:return>
                <xqdoc:type occurrence="*">node()</xqdoc:type>
            </xqdoc:return>


1.1.3 Annotations

            <xqdoc:annotations>
                <xqdoc:annotation name="rest:GET"/>
                <xqdoc:annotation name="rest:path">
                    <xqdoc:literal>"/test2"</xqdoc:literal>
                </xqdoc:annotation>
                <xqdoc:annotation name="rest:query-param">
                    <xqdoc:literal>"hello"</xqdoc:literal>
                    <xqdoc:literal>"{$hello}"</xqdoc:literal>
                    <xqdoc:literal>""</xqdoc:literal>
                </xqdoc:annotation>
            </xqdoc:annotations>

1.2 Custom Comment Tag

The @custom tag identifies a tag for any other purpose. If the @custom is followed immediately by a colon, then that value is in the tag attribute of the custom tag. e.g. @custom:openapi creates the tag <xqdoc:custom tag="openapi”></xqdoc:custom>

I am working on an XQuery script that will take information from the generated xqDoc from the src/main/ml-modules/services/*.xqy files and generate OpenAPI and RAML documentation.

NOTE: All comment tags allow for multiple lines of text.


2.0 MarkLogic ML Gradle additions to a build.gradle

The following is a sample build.gradle file.  If you add the contents of https://github.com/lcahlander/marklogic-xqdoc-display <https://github.com/lcahlander/marklogic-xqdoc-display> into src/main/ml-modules/root/xqDoc, then you will be able to view the xqDoc documentation of your XQuery code (http://localhost:8011/xqDoc <http://localhost:8011/xqDoc>).  A sample of the display is at: http://xqdoc.org/enhanced/default.html <http://xqdoc.org/enhanced/default.html>

gradle generateXQDocs
gradle importXQDoc


import org.apache.tools.ant.filters.BaseFilterReader

buildscript {
    repositories {
        jcenter()
    }

    dependencies {
        classpath "org.xqdoc:xqdoc:1.9.3"
    }
}

plugins {
  id "net.saliman.properties" version "1.4.6"
  id "com.marklogic.ml-gradle" version "3.6.0"
}

repositories {
  jcenter()
  maven { url "http://developer.marklogic.com/maven2/" }
  maven { url "http://repository.cloudera.com/artifactory/cloudera-repos/" }
}

configurations {
  mlcp {
    resolutionStrategy {
      force "xml-apis:xml-apis:1.4.01"
    }
  }
}

dependencies {
    mlcp "com.marklogic:mlcp:9.0.6"
    mlcp files("marklogic/lib")
}

class XQDocFilter extends BaseFilterReader {
    XQDocFilter(Reader input) {
        super(new StringReader(new org.xqdoc.MarkLogicProcessor().process(input.text)))
    }
}

task generateXQDocs(type: Copy) {
  into 'xqDoc'
  from 'src/main/ml-modules'
  include '**/*.xqy'
  rename { it - '.xqy' + '.xml' }
  includeEmptyDirs = false
  filter XQDocFilter
}

 task importXQDoc(type: com.marklogic.gradle.task.MlcpTask) {
  classpath = configurations.mlcp
  command = "IMPORT"
  database = "emh-accelerator-content"
  input_file_path = "xqDoc"
  output_collections = "xqdoc"
  output_uri_replace = ".*xqDoc,'/xqDoc'"
  document_type = "mixed"
}

2.1 eXist-db equivalent

The equivalent call would be
new org.xqdoc.ExistDBProcessor().process(input.text)

3.0 Issues

3.1 XQuery 3.1 extensions

The current grammar file is not properly parsing the following two items:

String Constructors  https://www.w3.org/TR/xquery-31/#id-string-constructors <https://www.w3.org/TR/xquery-31/#id-string-constructors>

URI Qualified Name https://www.w3.org/TR/xquery-31/#doc-xquery31-URIQualifiedName <https://www.w3.org/TR/xquery-31/#doc-xquery31-URIQualifiedName>

I need some help with the XQueryParser.g4 and XQueryLexer.g4 files to properly include those.  I have a question posted to StackOverflow on the String Constructors  https://stackoverflow.com/questions/53462040/i-am-having-trouble-translating-an-ebnf-grammar-to-an-antlr4-grammar <https://stackoverflow.com/questions/53462040/i-am-having-trouble-translating-an-ebnf-grammar-to-an-antlr4-grammar>

3.2 Implementation specific extensions

I believe that I have al of the eXist-db and MarkLogic language extensions implemented, but I need help testing the parser.

I still need to add the BaseX and Saxon extensions.


4.0 Future

I would appreciate the help from any that would like to contribute to the development
Once the following issues are resolved, then I am planning on coming out with a 2.0 release.
Add the display of the comments in markdown in the xqDoc display code
I am also interested in if anyone would be interested in helping me with  MarkLogic XQuery Code Scan using SonarQube utilizing the XQuery grammar and possibly the xqDoc files.