Re: Spec parser

Hi Tobie,

while the parser uses a hand full of Shepherd classes to access the DB, it's actually already pretty well decoupled from the rest of the system. The spec data is stored in a separate DB and has no dependencies on the rest of shepherd (the admin UI is part of Shepherd's admin UI, but that's a trivial part).

Note that there's a JSON API to get all the basic spec data from Shepherd:

http://test.csswg.org/shepherd/api/spec		- gets you a list of all specs known to the system along with basic data
and takes the arguments in any combination:
?spec=<spec-shortname>   - get data on only this spec
&sections=1  - get data on spec(s) sections
&anchors=1  - get data on all anchors in spec(s)

As well as a test coverage API that includes the specification statistics as well as the number of tests mapped to each anchor:
http://test.csswg.org/shepherd/api/coverage  (same arguments as the spec API)

(Note that the number of tests mapped to a spec anchor in the coverage API is the only bit that's dependent on the rest of Shepherd)

The parser also reads spec drafts and presents a union of sections and anchors between the official publication and the current draft (draft sections and anchors are flagged). It refreshes all official drafts every night and has a commit hook to refresh the CSS drafts after each push to our draft repository.

There's nothing about it that's particular to CSS specs, I can add any and all other specs to the CSSWG instance (it already parses HTML5 and SVG 1.1).

It wouldn't be very hard to make a stand-alone spec manager web app if you want a canonical instance running somewhere in W3C space, it would also be trivial to install a full Shepherd instance and simply not use the rest of it… (Shepherd is pretty much self-installing and self-maintaining these days, you just need a LAMP stack and a handful of Python libraries, like html5lib). I'd be happy to help with either.

Peter

On May 28, 2013, at 1:14 AM, Tobie Langel wrote:

> Hi Peter, hi folks,
> 
> I'm working on improving and automating our spec coverage tool[1] and Robin pointed out that Shepherd already had a pretty efficient spec parser[2].
> 
> Problem is it's tightly coupled with the rest of the Shepherd infrastructure.
> 
> Are there any plans to make this component standalone? I wouldn't mind giving it a go myself, but tests and docs are scarce.
> 
> Thanks,
> 
> --tobie
> ---
> [1]: https://github.com/w3c/web-platform-tests/tree/master/tools/coverage
> [2]: http://hg.csswg.org/dev/shepherd/file/58296977f3ee/python/SynchronizeSpec.py
> 
>

Received on Tuesday, 28 May 2013 16:09:40 UTC