RE: Submit Annotation test results - RE: PR of consolidated model tests created

Hi Tim, Ivan,

Thank you for the information!

Having a bit more time to test would actually be great for us as we envision already some challenges on our side due to some prescriptions we make in our implementation and some additional parameters in the requests due to authentication... but anyway, its a good opportunity for us to perform more tests and we will report back our results or any issues we may find along the way.

Best regards,


________________________________
From: Ivan Herman [ivan@w3.org]
Sent: 20 September 2016 19:23
To: Tim Cole
Cc: Hugo Manguinhas; W3C Public Annotation List
Subject: Re: Submit Annotation test results - RE: PR of consolidated model tests created


On 20 Sep 2016, at 18:16, Timothy Cole <t-cole3@illinois.edu<mailto:t-cole3@illinois.edu>> wrote:

Hugo-

Great to hear about your progress and interest in testing.  Please let me know if you run into issues with regard to annotation-model testing or with regard to submitting your test results.

As of right now, CR is scheduled to end 30 September.  However, the working group has been extended, and given the delays and issues in getting the annotation-model tests up and available (and in fact annotation collection and annotation page tests are still not up), I anticipate we may (likely will) have to announce at least a short extension in the CR testing period in order to meet CR exit requirements.

As for exactly what happens after CR exit criteria are met, I'm not entirely certain. Ivan or Shane may be able to advise better (both are at TPAC).  I'm fairly certain that the tests will remain available and online after CR exit.

That is correct.

I do not know whether the reports will be automatically updated or whether this is a manual process. Shane knows…

Ivan

  I know that the json schemas for testing against model requirements and recommendations will be available after CR in our test repository.  (We'll have more about how to download and use locally posted to the https://github.com/w3c/web-annotation-tests repository in a couple of weeks, but feel free to make use of them now.) Also, we have as you know an existing Wiki page listing implementations [1]. (Page is sparse and needs updating, in particular more links. Feel free to add to it.)

But in terms of being listed as one of the implementations counted towards CR, etc., I'm not sure what the protocol is regarding timing to be counted.

Thanks,

Tim Cole

[1] https://www.w3.org/annotation/wiki/Implementations



From: Hugo Manguinhas [mailto:Hugo.Manguinhas@europeana.eu]
Sent: Tuesday, September 20, 2016 9:04 AM
To: t-cole3@illinois.edu<mailto:t-cole3@illinois.edu>; 'W3C Public Annotation List' <public-annotation@w3.org<mailto:public-annotation@w3.org>>
Subject: RE: Submit Annotation test results - RE: PR of consolidated model tests created

Dear Tim, all,

Thank you for making the test suites and framework available!

It is great to have some external tests besides our own to validate our implementation, and we would very much like to submit the results and hopefully be mention as one of the implementations. With regards to timing, we will start setting up the tool and running the tests already this week, but to help us better manage our work we were wondering if there is a concrete deadline  for them to be considered for CR status... or if once the minimum of 2 implementations for each function is reached, the CR is approved, but implementations may continue to submit their results after that and still be listed as implementations.

Best regards,

Hugo Manguinhas
Technical R&D Coordinator

T: +31 (0)70 314 0998
M:
E: hugo.manguinhas@europeana.eu<mailto:hugo.manguinhas@europeana.eu>
Skype: hugo.manguinhas


Be part of Europe's online cultural movement - join the Europeana Network Association: http://bit.ly/NetworkAssociation<https://urldefense.proofpoint.com/v2/url?u=http-3A__bit.ly_NetworkAssociation&d=CwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=G5kQ0ct-0_9iF_vQMrCAQkzUzlX1tSFz982UNqicMEE&s=2DlwoIV6xguRhJIZ9tO3jVkw4eaXxHCyq9C-x2BxMVQ&e=>

 #AllezCulture!
Disclaimer: This email and any files transmitted with it are confidential and intended solely for the use of the individual or entity to whom they are addressed. If you have received this email in error please notify the system manager. If you are not the named addressee you should not disseminate, distribute or copy this email. Please notify the sender immediately by email if you have received this email by mistake and delete this email from your system.
________________________________
From: Tim Cole [t-cole3@illinois.edu<mailto:t-cole3@illinois.edu>]
Sent: 15 September 2016 03:40
To: 'W3C Public Annotation List'
Subject: Submit Annotation test results - RE: PR of consolidated model tests created
Just to follow up a bit,

If you have an annotation client, tool or service that creates json-ld serializations of annotations conformant to sections 1-4 of the Web Annotation Data Model, Candidate Recommendation, please run an annotation(s) representative of your application through our test suite and submit your results to the W3C test results server as described in the README.md file here:

  https://github.com/w3c/test-results/blob/gh-pages/annotation-model/README.md<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_w3c_test-2Dresults_blob_gh-2Dpages_annotation-2Dmodel_README.md&d=CwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=G5kQ0ct-0_9iF_vQMrCAQkzUzlX1tSFz982UNqicMEE&s=o08Lw6w5bes6b8FIdP9NqUd5QJRdTK49J5xvrHp5vEg&e=>

We can't exit CR successfully until we have demonstrated implementations, i.e., until we have enough test results demonstrating successful implementation of model features.

To facilitate your development efforts and the ongoing work some of you are doing to update from the older OA data model, you can also use the test suite to validate and confirm features implemented without submitting your results until your client has reached level of maturity desired. The URL for the annotation data model test suite is:

  http://w3c-test.org/tools/runner/index.html?path=/annotation-model/annotations/<https://urldefense.proofpoint.com/v2/url?u=http-3A__w3c-2Dtest.org_tools_runner_index.html-3Fpath-3D_annotation-2Dmodel_annotations_&d=CwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=G5kQ0ct-0_9iF_vQMrCAQkzUzlX1tSFz982UNqicMEE&s=01QnBMyFcHYkVVMwazqnmv4AnmrbYHHwdp4n2al870M&e=>

(As compared to Shane's URL, I've included the extra sub-folder in anticipation of the annotation collection and pagination tests still to be added in a different sub-folder).

The 3 sets of assertions included in the test suite can also can be accessed individually; the first one in particular focuses on validation:

  http://testdev.spec-ops.io:8000/annotation-model/annotations/annotationMusts-manual.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__testdev.spec-2Dops.io-3A8000_annotation-2Dmodel_annotations_annotationMusts-2Dmanual.html&d=CwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=G5kQ0ct-0_9iF_vQMrCAQkzUzlX1tSFz982UNqicMEE&s=Vwq7do2rZ5YWf2Q2YL-9DF_3Qnslg7OqvhD_TNYs7wg&e=>
  http://testdev.spec-ops.io:8000/annotation-model/annotations/annotationOptionals-manual.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__testdev.spec-2Dops.io-3A8000_annotation-2Dmodel_annotations_annotationOptionals-2Dmanual.html&d=CwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=G5kQ0ct-0_9iF_vQMrCAQkzUzlX1tSFz982UNqicMEE&s=Dt2QKcbZ6psh1rDZCN80-tiGXsTlO_QrjA59j6ni4vg&e=>
  http://testdev.spec-ops.io:8000/annotation-model/annotations/annotationsAgentOptionals-manual.html<https://urldefense.proofpoint.com/v2/url?u=http-3A__testdev.spec-2Dops.io-3A8000_annotation-2Dmodel_annotations_annotationsAgentOptionals-2Dmanual.html&d=CwMFAw&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=G5kQ0ct-0_9iF_vQMrCAQkzUzlX1tSFz982UNqicMEE&s=J-nxgTglfmy7ZXK53H8FTUddkiS2Bpm4rhUtOzT0VZo&e=>

Thanks for your help,

Tim Cole
University of Illinois at Urbana-Champaign

From: Shane McCarron [mailto:shane@spec-ops.io]
Sent: Wednesday, September 14, 2016 9:53 AM
To: W3C Public Annotation List <public-annotation@w3.org<mailto:public-annotation@w3.org>>
Subject: Re: PR of consolidated model tests created

Congratulations everyone on the work on this - it is now integrated into WPT!  People can access the formal version of the tests at [1].  My development server will continue to be available if we need to do further refinement.

[1] http://w3c-test.org/tools/runner/index.html?path=/annotation-model<https://urldefense.proofpoint.com/v2/url?u=http-3A__w3c-2Dtest.org_tools_runner_index.html-3Fpath-3D_annotation-2Dmodel&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=4vxEhtxQQXN1ogCVlM9mrdKxeJQ1vnRta2e24yQbB84&s=6wNHVuydw51sk75ADfbuUaGX-PYagLoZhQrkuLsqTQU&e=>

On Tue, Sep 13, 2016 at 11:18 AM, Shane McCarron <shane@spec-ops.io<mailto:shane@spec-ops.io>> wrote:
Please review the PR at [1]. It reflects what was discussed on list and at the 9 Sep meeting.  Comment on the PR if you agree with it so the WPT powers that be will do the merge.

[1] https://github.com/w3c/web-platform-tests/pull/3704<https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_w3c_web-2Dplatform-2Dtests_pull_3704&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=4vxEhtxQQXN1ogCVlM9mrdKxeJQ1vnRta2e24yQbB84&s=2okUv5-t9RYC_OSTwi0GjBdRsxLLsR0Nm6hpumDMcrc&e=>

--
Shane McCarron
Projects Manager, Spec-Ops



--
Shane McCarron
Projects Manager, Spec-Ops


----
Ivan Herman, W3C
Digital Publishing Lead
Home: http://www.w3.org/People/Ivan/
mobile: +31-641044153
ORCID ID: http://orcid.org/0000-0003-0782-2704