RE: annotation model testing issue over the weekend

Shane-

Something a little odd with regard to most recent PR for the annotation-model testing. If you are still are at TPAC, maybe you can check into it? Or maybe it will straighten itself out next week after TPAC.

w3c-test.org server seems to be out of sync with https://github.com/w3c/web-platform-tests repository, at least with regard to most recently updated file in /annotation-model/definitions/ folder.

PR #3766 was merged into the web-platform-tests repository, but this change is not visible as of 10 AM Central today (as best I can tell) in w3c-test.org.  Compare:

  http://www.w3c-test.org/annotation-model/definitions/specificResource.json

to

  https://raw.githubusercontent.com/w3c/web-platform-tests/master/annotation-model/definitions/specificResource.json

and to the changes shown for PR #3766 in the commit

 https://github.com/w3c/web-platform-tests/commit/0602b49539785ade9dde966ebbee8f3ed2546b94

The first change to the specificResource file was the renaming of a sub-schema in the file from selectorDetected to selectorDefined and the insertion of a new much shorter selectorDetected sub-schema.  When I looked a few minutes ago these changes are not showing up in the w3c-test.org version of the file even though they are present in the web-platform-tests repo.

As a result still getting schema references not found for some of the test assertions.

I tried from a couple of different browsers and physical clients to make sure I wasn't being fooled by some caching behavior.

If you find anything out about this, let me know, or maybe latency will straighten out on its own

Thanks,

Tim Cole

________________________________
From: Shane McCarron [shane@spec-ops.io]
Sent: Wednesday, September 21, 2016 05:52
To: Cole, Timothy W
Cc: W3C Public Annotation List
Subject: Re: annotation model testing issue over the weekend

These changes should now be pushed to w3c-test.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__w3c-2Dtest.org&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=rGcNWelFDrXlH1ReG1Qw1bJ8ZlWWCIxR92X1OXbt1hA&s=_j2wWjDU0v32Isb2TOGW5EwVetKsgqxItkUX3nhy6zs&e=> too.

On Tue, Sep 20, 2016 at 11:50 AM, Timothy Cole <t-cole3@illinois.edu<mailto:t-cole3@illinois.edu>> wrote:
For any of you have been testing annotations against the annotation-model tests during the course of the last 5 days, you will have noticed some issues.  the testdev.spec-ops.io:8000<https://urldefense.proofpoint.com/v2/url?u=http-3A__testdev.spec-2Dops.io-3A8000&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=rGcNWelFDrXlH1ReG1Qw1bJ8ZlWWCIxR92X1OXbt1hA&s=XckmHUvTb1kgOOx_M9TH_LulJmTKtpBCZ1xPDzfifM0&e=> server was down for a few days and the w3c-test.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__w3c-2Dtest.org&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=rGcNWelFDrXlH1ReG1Qw1bJ8ZlWWCIxR92X1OXbt1hA&s=_j2wWjDU0v32Isb2TOGW5EwVetKsgqxItkUX3nhy6zs&e=> server has been throwing schema not found kinds of errors for about 15% of the annotation model assertions being tested.  We talked about the testdev server issue during Friday's call, but I didn't detect the w3c-test.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__w3c-2Dtest.org&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=rGcNWelFDrXlH1ReG1Qw1bJ8ZlWWCIxR92X1OXbt1hA&s=_j2wWjDU0v32Isb2TOGW5EwVetKsgqxItkUX3nhy6zs&e=> issue until yesterday. My apologies.

The testdev server is up again, so you should be good to run your tests on that server today and subsequent:

   http://testdev.spec-ops.io:8000/tools/runner/index.html?path=/annotation-model/annotations/<https://urldefense.proofpoint.com/v2/url?u=http-3A__testdev.spec-2Dops.io-3A8000_tools_runner_index.html-3Fpath-3D_annotation-2Dmodel_annotations_&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=rGcNWelFDrXlH1ReG1Qw1bJ8ZlWWCIxR92X1OXbt1hA&s=yg-JtafY0nyQwso30rcQWL1n-ehMLqzCXRSawsdPMfQ&e=>

A PR has been submitted for the repository behind the w3c-test.org<https://urldefense.proofpoint.com/v2/url?u=http-3A__w3c-2Dtest.org&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=rGcNWelFDrXlH1ReG1Qw1bJ8ZlWWCIxR92X1OXbt1hA&s=_j2wWjDU0v32Isb2TOGW5EwVetKsgqxItkUX3nhy6zs&e=> server that will fix the missing schema errors.  It has been reviewed by Benjamin and me, and so should be merged soon (noting that many of those who can merge are at TPAC).  Once the latest PR has been merged, and once I've more carefully validated that all annotation-model tests are working, I'll send a follow-up email.

NB for anyone testing annotations: Other implementers have mentioned that it is important to wait for each test page to fully load before pasting your json-ld into the text box on the page.  The test page is fully loaded when the list of assertions being tested by that page appears below the text box into which you paste your json-ld.

Sorry again for any inconveniences encountered over the weekend or yesterday.

Thanks,

Tim Cole
University of Illinois at Urbana-Champaign




--
Shane McCarron
Projects Manager, Spec-Ops