Compat Table Progress

Hi, folks–

I'm very pleased to announce that we've passed a major milestone in the 
compatibility table project!

This week, Renoir converted the MDN compatibility table information into 
a normalized JSON file.

As I understand it, he used Frozenice's MDN-to-JSON crawler script [2] 
to do the first pass, then wrote another script [3] to normalize the 
data incrementally (in occasional consultation with me). This was 
time-consuming, since MDN's data was rather inconsistent both in 
formatting and in how the results were reported, as well as having quite 
a bit of ambiguity. He recorded some of these anomalies [4], and 
preserved them in a "notes" entry for that browser/feature, so we 
wouldn't lose the data, while making his best guess for the normalized 
version of the data; we hope that this cleaned-up data will be useful 
for the MDN project's own compat-data project, and that we can work 
together with MDN to make the data even better over time.

Even though MDN's data is not perfect (it's often out of date, and 
reporting is spotty on some browsers for some features), it's a great 
starting point for our own compat data for a few reasons:
* MDN was generous enough to share the data with us!
* it's organized in a way very similar to ours (no coincidence, really, 
since we used MDN as inspiration for how we organized WPD)
* it often goes into great detail about sub-features
* it has the widest coverage for older features (CanIUse mostly focuses 
on newer stuff)
* it was available via their API (QuirksMode's data is still locked up 
in HTML tables, and we'll need to help PPK convert that)

We decided to structure the data to follow the JSON data model (and 
result coding [5] from CanIUse, for a couple of reasons:
* it's a proven model, and we didn't have to finalize the details of our 
own more comprehensive data model
* it was simple to convert to that format
* it will help us integrate the CanIUse data (which is one of our next 
steps)
* it is largely compatible with the existing MediaWiki extension that I 
wrote all those months ago.

As you know, reporting compatibility information is one of the major 
roadblocks to our announcement of the CSS properties, and importing the 
MDN was definitely one of the more challenging and time-consuming parts; 
the enormity of the task scared away more than one contributor! So I 
have to give a huge shout-out to Frozenice for writing the original 
script, Pat Tressel for helping out in the discussions and framing the 
problem, and especially Renoir for diligently normalizing the data and 
producing the final output!

The next step is to finalize the MediaWiki extension to push the data 
into our pages, which I hope will not be hard.

(BTW, both Frozenice and Renoir wrote the scripts in NodeJS, which is 
what we're going to try to write most of our future code in.)


[1] 
https://raw.github.com/webplatform/mdn-compat-importer/internal-js-object/data/compat-mdn.json
[2] https://github.com/webplatform/mdn-compat-importer
[3] 
https://github.com/webplatform/mdn-compat-importer/blob/internal-js-object/lib/EntityConverter.js
[4] 
https://github.com/webplatform/mdn-compat-importer/blob/1d633d4d42cf622c30981484b182c4e7aa507262/data/compat-anomalies.txt
[5] https://github.com/Fyrd/caniuse/blob/master/Contributing.md

Regards-
-Doug

Received on Friday, 7 February 2014 16:58:54 UTC