Re: Shepherd

> On Aug 19, 2015, at 11:01 PM, Anne van Kesteren <annevk@annevk.nl> wrote:
> 
> Peter, thank you for all this detailed information.
> 
> On Wed, Aug 19, 2015 at 9:09 PM, Linss, Peter <peter.linss@hp.com> wrote:
>> (calling it without any arguments just lists the specs available, by passing args you can get detailed information on all anchors in the specs.)
> 
> I see.
> 
> Now I it's clear that everything is available publicly I guess what
> I'm mostly looking for is some way to report bugs.

I did set up a bug tracker years ago, but never really published it, FWIW it’s at:
https://dev.csswg.org/projects/shepherd <https://dev.csswg.org/projects/shepherd>
I haven’t even looked at it myself in quite a while… not sure all the issues there are still current or relevant.

These days it’s best to just ping me (or Tab) via email or IRC. Relatively simple things get fixed pretty quickly.

Tab has admin access to the spec db as well for adding new specs or updating URLs and such.

> The nice thing
> about https://github.com/tobie/specref is that the database is in
> GitHub and you can inspect it and report issues against it. Even
> though the database is fully automated, that still makes it easier to
> inspect things manually.

Well, the spec DB contains all the anchors and a bunch of metadata about each (like what each <dfn> defines), the anchor table in MySQL is currently at 50MiB and gets updated several times per day. I could set up a dump to GitHub, but it’s going to get large… not sure that’ll help sift through it.

If you just want a list of the specs indexed (and their metadata) that would be considerably smaller, but there are a bunch of tools to help visualize the JSON dump from the API...

> 
> E.g., it's unclear to me whether Shepherd indexes all WHATWG Standards
> or whether some cannot be indexed currently.

The list of specs that it indexes is manually entered, if we’re not indexing all of the WHATWG specs it’s just because we haven’t added them yet. I’m happy to add any that are missing. There shouldn’t be any reason why they can’t be indexed (it also handles multipage specs fine), at a minimum it’ll find all the anchors, section headings and parse the WebIDL. It may or may not identify the types of all the <dfn> anchors, depending on how they’re marked up (it understands Bikeshed markup just fine and has some heuristics for older specs like CSS2.1, SVG, and HTML5).

The one thing it won’t parse is Respec source, so if you have Respec specs, it needs a URL of the Respec output (the parser is written in Python and can’t run js client-side).

> It's unclear what
> shortname they've been assigned

For W3C specs we use the /TR shortname, for WHATWG we just pick something, happy to change them or have you tell us what you want there.

> and why e.g., "dom" doesn't get HTTPS
> URLs.

I believe it does now (we just changed it yesterday).

> 
> Having it all somewhere as a dump on GitHub or some such would make
> that a bit more visible.

We are planning on making a general index page at some point in the not too distant future. Something that will use the anchor DB to create a master index of all the CSS properties, HTML & SVG elements and attributes, and all the IDL constructs, linking back to the specs where they’re all defined. Having that will definitely put any omissions or errors right out front as well.