Re: On deploying next release of compatibility tables

Hi all,

Frozenice is right, the line wpd.data[ec.getSlug()] = row; was
effectively rewriting previous members and its my fault :( 

TL;DR, the changes aren’t big and it won’t be too long to put in place,
some work is done (see "What is done so far"). But I still have a few
concerns (see "Current concerns") to address before completing tasks.

Provided we agree on how to address my concerns this week, I expect to
be able to announce the rollout on the test wiki by the end of the week.




*What has to be done*

After a conversation with Doug, we decided to add two new work items
before deploying on the live wiki.

Hopefully those will address the problem (see explaination in "Problem
in the current schema").

Tasks:

1. Adapt compat data schema by creating "subfolders" (e.g. will be more
precise on our choice)
2. Adapt CompaTables MediaWiki extension to match that schema change

The schema change is almost complete (see "What is done so far"), but I
will need to discuss about a few details (see "Concerns").




*Problem in the current schema*

In the current model [1], to access a table contents (e.g. 'script'), we
are reading from a similar object:
   
    var compat = {data: {script: {contents: {desktop:{},mobile:{}},
links:[]}}};

The CompaTables extension would we would search for $keyword in the
Object like this:

    compat > data > $keyword > contents

And generate the two tables (mobile, desktop) based on what’s there.

The markup in the test wiki is currently:
   
    <compatability feature="script" format="table"></compatability>

The problem is that the scraper wasn’t looking for duplicates and was
writing in 'compat > data'.

At the time of writing, there were about 12 properties that was
overwritten. Its not a big problem, but we are losing data.

In consequence, we have no ways to know if we were reading the right
data. This is the problem we are hoping the schema change will address
(see 'The schema change').




*The schema change*

Instead of adding elements directly in 'data', we create subfolders.

Like so:

  var compat = {
        data: {
          css: {},
          element: {
            script:{
              contents:{desktop:{}, mobile:{}},
              links:[]
            }
          },
          svg: {
            script:{
              contents:{desktop:{}, mobile:{}},
              links:[]
            }
          }
          /* figure out other subfolders here TODO */
        }
     };

To access $keyword (e.g. 'script') in $parent (e.g. 'element') the
extension will have to dig in the Object this way:

    compat > data > $parent > $keyword > contents

... and the wiki tag would have a new parent="..." attribute.

    <compatability feature="script" parent="element"
format="table"></compatability>

I’ve started someting (see "What is done so far", "Latest run"), but I
have a few details to adjust with you guys (see "Current concerns").



*Current concerns*:

- Which subfolders we want for the new schema
- Figure out a way, to detect and sort items (see "Latest run")




*What is done so far*

What you see in the latest run [0] is what we can get from MDN based on
the URI is not perfect but its a starting point.

What it does:
- Makes sure it doesn’t overwrite
- If a member already exist, send in a temporary array ("duplicates")
- Added a "jsonselect" property showing where the element is sorted in

Have a look at [0], and see how some duplicates gets sorted:
 - "script"
 - "a"

At this time, I have to discuss with you about how we figure out in
which subfolder a given object goes. Its currently using the original
URI from MDN but its not what we need (see "Latest run"). Also, we will
most likely not sort the same way as that URI.




*Latest run*

On the latest run, at [0], you can see the following subfolders:
  - css
  - element
  - attribute
  - api
  - web
  - webapi
  - events
  - global-objects
  - javascript-typed-arrays
  - operators
  - geolocation

I added some more details in the output.
- Each member has a 'jsonselect' property illustrating where that member is
- Temporary "duplicates" array in case we are missing ones
- Temporary "sections" showing the folders it creates



  [0]: http://docs.webplatform.org/compat/inprogress-human.json
  [1]: http://docs.webplatform.org/compat/compat-human.json
  [2]:
https://github.com/webplatform/mdn-compat-importer/commit/42fdb23ac245d786856fc52b2e89bbb573d30138#diff-49cd78ccff0d1a2daaea74ebcdc89e7eL241
 

-- 
Regards,

Renoir Boulanger  |  Developer operations engineer
W3C  |  Web Platform Project

http://w3.org/people/#renoirb  ✪  https://renoirboulanger.com/  ✪  @renoirb
~