- From: Timothy Cole <t-cole3@illinois.edu>
- Date: Thu, 23 Jun 2016 18:10:33 -0500
- To: "'W3C Public Annotation List'" <public-annotation@w3.org>
- CC: "'Discussions about Test Development'" <testdev@lists.spec-ops.io>
- Message-ID: <075301d1cda4$78ee9f90$6acbdeb0$@illinois.edu>
Getting back to referencing of external schema in the WPT environment, e.g., referencing definitions that are needed by multiple top-level schema, I think strategy 2 from your 8 June email will work best. As compared to referencing by absolute URIs (option 1), approach 2 makes it much easier to develop and test schemas offline, etc. Our local experimentation with ajv also suggests there is the potential to run into security issues when referencing by URI.
HOWEVER, we note that there appears to be a caveat or two. In particular, how is likely more obvious to those more familiar with json schema and json pointer than it was to me initially, schema id impacts how $ref is resolved. Because the same definition schema may be referenced from multiple locations in the folder hierarchy, we've found that the most flexible method that seems to work with ajv is if the value of id within all schemas is just the filename with no path component.
So for example, the following schema has been added to /annotation-model/definitions
{
"$schema": "http://json-schema.org/draft-04/schema#",
"id" : "stringUri.json",
"type": "string",
"format": "uri"
}
Assuming file location is provided to ajv during initialization -- e.g., for agv-cli you can add the command line argument, -r "definitions/*.json" (or equivalent depending on relative path to the definitions folder) or for agv API you can use .addSchema or similar – ajv seems happy to allow subsequent references to id value without the need to provide path components, e.g.,
…
"target":
{"oneOf": [
{ "type" : ["string", "object"],
"anyOf": [
{"$ref": "stringUri.json#" },
{"$ref": "choiceSet.json#" },
{"$ref": "specificResource.json#" },
{"$ref": "externalWebResource.json#" }
]
}
…
Or at least this works well for building and testing schema locally in my environment. This approach makes it easy to reference the definitional schema from multiple locations without having to worry about differences in path relative to the referencing schema. It is also relatively easy to include sub-schema within your external file when that makes sense. Will this approach transfer to the WPT environment easily?
Janina added 5 definition schemas to the https://github.com/w3c/web-annotation-tests/tree/master/definitions folder, and she's added 2 schemas to the /annotations folder that reference these definitions. For what it's worth, note also that /definitions/externalWebResource.json references /definitions/stringUri.json, illustrating the advantages of having no path information in the id.
Next step will be to create a test that implements at least one of the new version schemas in the /annotations folder and then verify that test runs correctly and without invalid reference errors.
Unrelated heads up, as we add the schemas called for in the spreadsheet discussed (briefly) on last WG call, most of the schemas currently in the /common folder will be revised and migrated to the /annotations folder. Truly common schemas will then be put in the /common folder. This migration will likely break one or both of the sample tests in the /framework/annotations folder. So will wait to delete the old schemas in /common until we've updated the test scripts.
Thanks,
Tim Cole
From: Testdev [mailto:testdev-bounces@lists.spec-ops.io] On Behalf Of Shane McCarron
Sent: Wednesday, June 08, 2016 4:30 PM
To: Discussions about Test Development <testdev@lists.spec-ops.io>
Cc: W3C Public Annotation List <public-annotation@w3.org>
Subject: Re: [Testdev] [model-test] referencing external schema
Okay. We have two strategies in mind:
1) have a top level definitions folder, then use absolute URIs (e.g., /annotation-model/definitions/my_def.json)
2) have that top level folder, and have ajv automatically import all of the .json files in that folder during initialization.
Would either of those work for you Tim?
On Wed, Jun 8, 2016 at 2:50 PM, Timothy Cole <t-cole3@illinois.edu <mailto:t-cole3@illinois.edu> > wrote:
Yes. I actually had created the definitions folder before I realized relative addressing wasn't working. I'm in the process of creating some better examples of re-useable definitions – the samples I included in my email are out of date and will be replaced.
Thanks,
Tim Cole
From: Testdev [mailto:testdev-bounces@lists.spec-ops.io <mailto:testdev-bounces@lists.spec-ops.io> ] On Behalf Of Benjamin Young
Sent: Wednesday, June 08, 2016 2:26 PM
To: Discussions about Test Development <testdev@lists.spec-ops.io <mailto:testdev@lists.spec-ops.io> >
Cc: W3C Public Annotation List <public-annotation@w3.org <mailto:public-annotation@w3.org> >
Subject: Re: [Testdev] [model-test] referencing external schema
Hey Tim,
Sorry for the wait. Shane and I just chatted about this $ref thing and I think we can make it work in WPT. It’s mostly a question of what the relative path expectations of `ajv` tucked inside of WPT might be.
It looks like we may also be able to hack around the defaults (if need be) using wptserve’s Pipes system:
<https://urldefense.proofpoint.com/v2/url?u=https-3A__wptserve.readthedocs.io_en_latest_pipes.html&d=CwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=CKlycpO63cn6OBnNNc-wEYw8fUsLLS2T3j_BvQUxqM4&s=yKFYr73mCj5fVvAFd-pW7tFFDalI4fVfaycO-20g66Q&e=> https://wptserve.readthedocs.io/en/latest/pipes.html
What we’re thinking right now is taking the JSON objects you now have in the `definitions` object (which you pasted below) and putting those inside of a `definitions` folder and then working through the relative path details with WPT.
Would that serve as a solution for what you’re attempting here?
Cheers!
Benjamin
From: Testdev [mailto:testdev-bounces@lists.spec-ops.io] On Behalf Of Shane McCarron
Sent: Wednesday, June 8, 2016 10:46 AM
To: Discussions about Test Development <testdev@lists.spec-ops.io <mailto:testdev@lists.spec-ops.io> >
Cc: W3C Public Annotation List <public-annotation@w3.org <mailto:public-annotation@w3.org> >
Subject: Re: [Testdev] [model-test] referencing external schema
Did you get any traction on this Tim?
On Thu, Jun 2, 2016 at 10:19 AM, Timothy Cole <t-cole3@illinois.edu <mailto:t-cole3@illinois.edu> > wrote:
We are currently developing json schemas to help test for implementations of SpecificResource features of the data model. I've uploaded 4 of the schemas currently being developed to https://github.com/w3c/web-annotation-tests/tree/master/SpecificResource <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_w3c_web-2Dannotation-2Dtests_tree_master_SpecificResource&d=CwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=CKlycpO63cn6OBnNNc-wEYw8fUsLLS2T3j_BvQUxqM4&s=V_xozOO0wxNGrG964fbg7j8vJDgiNPr-Zf3a3z7cUss&e=> (these are first drafts, not quite ready, still being debugged and refined). All 4 schemas share a definitions section containing 2 sub-schemas (appended below). These are easy to reference internally where needed, but it would be even more convenient to reference them as external schemas. This should be possible using $ref, but I've had difficulties getting this approach to work with my local ajv instance, especially with regard to relative file locations.
Is anyone familiar enough with how $ref works within the Web Platform Testing framework's implementation of ajv to advise?
Below is the current way the sub-schemas are embedded in each primary schema. Obviously this works, but it means repeating the same sub-schemas in multiple files, which is obviously less efficient than if these sub-schemas could be referenced as individual files.
Thanks,
Tim Cole
{
"definitions": {
"SpecificResource": {
"type": "object",
"properties": {
"selector": {},
"state": {},
"styleClass": {},
"renderedVia": {},
"scope": {},
"purpose": {}
},
"anyOf": [
{"required": ["selector"]},
{"required": ["state"]},
{"required": ["styleClass"]},
{"required": ["renderedVia"]},
{"required": ["scope"]},
{
"required": ["purpose"],
"not": {"required": ["value"]}
}
]
},
"SpecificResType" : {
"type": "object",
"properties": {
"type": {
"oneOf": [
{"type": "string",
"enum": ["SpecificResource"]},
{"type": "array",
"items": [{ "type": "string",
"enum": ["SpecificResource"] }]
} ]
}
},
"required": ["type"]
}
}
}
_______________________________________________
Testdev mailing list
Testdev@lists.spec-ops.io <mailto:Testdev@lists.spec-ops.io>
http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.spec-2Dops.io_listinfo.cgi_testdev-2Dspec-2Dops.io&d=CwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=CKlycpO63cn6OBnNNc-wEYw8fUsLLS2T3j_BvQUxqM4&s=AKojwtr16bHcWXtHtf8ZR9e_-lTF7Oan-BiiKomym3I&e=>
--
Shane McCarron
Projects Manager, Spec-Ops
_______________________________________________
Testdev mailing list
Testdev@lists.spec-ops.io <mailto:Testdev@lists.spec-ops.io>
http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.spec-2Dops.io_listinfo.cgi_testdev-2Dspec-2Dops.io&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=19fTLxSHQ7CJCtVk1Lcg9AHFHSM3jG803vzgqbR4H2E&s=YjGrRdor7B_zkXJ3Vd-wgMJitXS21c41ZVhzFla6TcY&e=>
--
Shane McCarron
Projects Manager, Spec-Ops
Received on Thursday, 23 June 2016 23:11:17 UTC