Re: {Minutes} TTWG Meeting 2020-10-01 - Consensus for resolutions following review period

   hi,

On 2020/11/19 20:28, Atsushi Shimono (W3C Team) wrote:
>> That's great, thank you Atsushi. I see that the process added back in a "previous version" link to the front matter, which we'd removed. That's a bit strange but not a huge problem. Any idea how that happened?
> 
> 
>    mm.. haven't noticed on that, also I have no idea on this, sorry.
>    output generated from W3CMANIFEST does not include that line:
> https://w3c.github.io/tt-profile-registry/index.html?specStatus=WG-NOTE&shortName=ttml-profile-registry
> but extracted metadata (in output from echidna) contains previousVersion. so, one possibility
> is that echidna adds options of previous* in addition to ones specified in W3CMANIFEST
> automatically...

   TL;DR, I think I've found lines of root cause in spec-generator and possible required change in
Specberus (and run-publisher, not sure), do we want to submit a change request?


----


   looking process in details, with code.

   API request to echidna will be received by lines:
https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/app.js#L212
   API application registration as /api/request, passing request part to processRequest():
https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/app.js#L100
   extracts URL parameter and pass to Orchestrator module without no modification at
   https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/app.js#L148
Orchestrator module receives parameters and execute list of steps:
https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/orchestrator.js#L493
   which passes URL parameter without modification to runDocumentDownloade() at:
   https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/orchestrator.js#L498
https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/orchestrator.js#L180
   which calls fetchAndInstall() method of DocumentDownloader module at:
   https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/orchestrator.js#L215
and, DocumentDownloader.fetchAndInstall() will run:
   https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/document-downloader.js#L74
   to download MANIFEST file, and handle its lines at:
   https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/document-downloader.js#L121
and, extract source of drafts into SPEC_GENERATOR URL at:
   https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/document-downloader.js#L137
   as, SPEC_GENERATOR?type=<2nd parameter>&url=<MANIFEST_URL + URL with specified parameter>

   tt-profile-registory has W3CMANIFEST as:
https://github.com/w3c/tt-profile-registry/blob/gh-pages/W3CTRMANIFEST
index.html?specStatus=WG-NOTE&shortName=ttml-profile-registry respec

   so, URL to be fetched via spec generator will be
SPEC_GENERATOR?type=respec&url=https://w3c.github.io/tt-profile-registry/index.html?specStatus=WG-NOTE&shortName=ttml-profile-registry
and, its' reachable end point in public is:
https://labs.w3.org/spec-generator/?type=respec&url=https%3A%2F%2Fw3c.github.io%2Ftt-profile-registry%2Findex.html%3FspecStatus%3DWG-NOTE%26shortName%3Dttml-profile-registry

   which inserts Previous Version header lines...

   In spec-generator code at: https://github.com/w3c/spec-generator
   passed target URL is parsed into URL object as specURL:
https://github.com/w3c/spec-generator/blob/51466a03745828bc69ee07be7f7decc1a0e5abfa/server.js#L59
   which extracts URL parameters
if shortName is specified, spec-generator will add previousMaturity and previousPublishDate:
   https://github.com/w3c/spec-generator/blob/51466a03745828bc69ee07be7f7decc1a0e5abfa/server.js#L71
and results into containing previous version header.

   After retrieval of spec text, extractMetadata() and validate() from Specberus will run:
extract: https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/orchestrator.js#L510
validate: https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/orchestrator.js#L305
   and validator loads profile of 'specberus/lib/profiles/TR/WG-NOTE-Echidna' for WG-NOTE ones:
https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/specberus-wrapper.js#L34
   whose extraction rules are:
https://github.com/w3c/specberus/blob/main/lib/profiles/TR/WG-NOTE.js
   which has a following line, and will require previousVersion.
> ,   previousVersion:    true

   For later process, token-checker (just check token passed to echidna), transition-checker
(do nothing for WG-NOTE, below), thrid-party-resources-checker (checks resources loaded, below),
and tr-installer (below, calling 'cp' command) seem not to use previousVersion.
transition-checker: https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/transition-checker.js#L33
thrid-party-resource-checker: https://github.com/w3c/third-party-resources-checker
tr-installer: https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/orchestrator.js#L26
   For update-tr-shortlink (pointing /TR/<short link>/ to new one) seems to run local command
only with thisVersion metadata, and not to use previousVersion.
https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/orchestrator.js#L472
https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/orchestrator.js#L41


   but run-publisher will post previousVersion:
https://github.com/w3c/echidna/blob/a1ae6e7499d929c25d865b26518018f95bd8a84b/lib/publisher.js#L43
and could return error on no previousVerwion situation.



   We might be possible to submit a change request of this behavior in spec-generator and possible
required change in Specberus (and run-publisher if related), and do we want to take an action?

Received on Tuesday, 24 November 2020 11:50:14 UTC