Re: TPAC demo video on Reffy

Hi Ivan,

Documentation is lacking a bit there.

Built-in processing modules are those that appear in:
https://github.com/w3c/reffy/blob/main/src/browserlib/reffy.json

Essentially, these are the modules that ship in Reffy today:

- css: CSS property definitions
- date: last modified date
- dfns: all term definitions (used to populate the cross-reference 
database)
- elements: definitions of markup elements (from HTML and SVG specs)
- generator: tells you whether the spec uses ReSpec, Bikeshed or some 
other tool
- headings: information about headings
- idl: Web IDL (used to analyze the IDL and maintain IDL tests in Web 
Platform Tests)
- ids: list all IDs available in the spec
- links: links to other specs along with the list of fragments targeted 
in these specs.
- refs: list of normative and informative references in the spec (from 
the "References" section)
- title: extracts the spec title

What gets generated is essentially what you will find under the relevant 
folders in:
https://github.com/w3c/webref/tree/main/ed

There is no built-in module that extracts all normative statements. You 
can have a look at the pull request I mentioned [4] for a possible 
starting point. Note that it is going to generate ~400MB of data if you 
run that across all specs! Also, it probably does not do what you'd like 
it to do ;-)

Apart from that, to create a processing module, all you need to do is 
create a file whose suffix is ".mjs" that exports a default function:

export default function() {
   // The function will run in a headless browser once the spec has been 
loaded
   // ...

   // The function needs to return something that can be serialized as 
JSON
   // (string, number, object, array)
   return "some outcome";
}

Hope that helps,
François.


------ Original message ------
From: "Ivan Herman" <ivan@w3.org>
To: "Francois Daoust" <fd@w3.org>
Cc: "spec-prod@w3.org" <spec-prod@w3.org>; "Dominique Hazaël-Massieux" 
<dom@w3.org>
Date: 08/10/2021 18:00:09

>Hi François,
>
>That looks interesting. However, how can I find the available 'built' 
>in modules that I can use out of the box? I am interested by that tool 
>that extracts all normative statements you refer to in the mail, is 
>that an existing module? If not, how do I find some information on how 
>I should create such a module?
>
>Thanks
>
>Ivan
>
>>On 8 Oct 2021, at 17:32, Francois Daoust <fd@w3.org> wrote:
>>
>>Hi spec fans,
>>
>>Reffy [1] is the spec crawling and processing tool that powers Webref 
>>data updates [2]. In turn, Webref is used to maintain Web IDL tests in 
>>Web Platforms Tests, Web IDL types in TypeScripts, and the 
>>cross-reference database in ReSpec.
>>
>>As part of TPAC demos released this week, I recorded a short video to 
>>showcase how Reffy can be used as a command-line interface tool, which 
>>I thought some of you might be interested to look at:
>>
>>  https://www.w3.org/2021/10/TPAC/demos/reffy.html
>>
>>The video intends to demonstrate how you may now tame Reffy to run a 
>>custom spec processing module of your own, so as to extract specific 
>>info from Web specifications. This can typically be used to answer 
>>questions such as:
>>
>>- How to find all spec editors in my company? (this was discussed on 
>>this mailing-list some time ago, see [3])
>>- Can I create useful extracts to map normative statements in specs to 
>>implementation code? (see discussion in [4])
>>- Which specs don't follow the [insert some automatically testable 
>>guideline here] guideline?
>>
>>It could also be used to compute plenty of useful or dubious 
>>statistics, such as:
>>- What is the average length of an abstract?
>>- How many sections does a table of contents typically have?
>>- How many links, examples, illustrations, notes, etc.?
>>
>>Let us know if you find bugs. Also, we'd love to know if you actually 
>>make use of it for something!
>>
>>Thanks,
>>Francois.
>>
>>[1] https://github.com/w3c/reffy
>>[2] https://github.com/w3c/webref
>>[3] 
>>https://lists.w3.org/Archives/Public/spec-prod/2021AprJun/0026.html
>>[4] https://github.com/w3c/reffy/pull/153
>>
>>
>
>
>----
>Ivan Herman, W3C
>Home: http://www.w3.org/People/Ivan/
>mobile: +33 6 52 46 00 43
>ORCID ID: https://orcid.org/0000-0003-0782-2704
>

Received on Friday, 8 October 2021 17:25:59 UTC