- From: Shane McCarron <shane@spec-ops.io>
- Date: Thu, 9 Jun 2016 10:58:49 -0500
- To: Discussions about Test Development <testdev@lists.spec-ops.io>
- Cc: W3C Public Annotation List <public-annotation@w3.org>
- Message-ID: <CAJdbnOD1V0vaf2s+DYXs_fvb39AWamQHZM3CYo8XS92rLxXgFw@mail.gmail.com>
Typically WPT folder trees are designed to mirror the structure of the spec
being tested. That way it is easy to correlate the assertions with the
content of the spec. Beyond this, there are sometimes top level folders
like "tools" or "common". I think it is important that we stick with that
sort of structure to help with adoption by the wpt community.
On Wed, Jun 8, 2016 at 5:37 PM, Timothy Cole <t-cole3@illinois.edu> wrote:
> I think either approach should work, but I've had some issues even with
> referencing absolute URIs . The ajv import approach might be a little
> easier and more reliable? Regardless, now that the patterns are becoming
> clear, copying definitions to multiple schemas not all that onerous. You
> can always include definitions in a schema and not reference… Obviously
> the big issue is when you want to revise a definition (or upgrade to schema
> version 5). Then having to go back and fix in multiple files would indeed
> be an issue. So hopefully one of your approaches will work.
>
>
>
> To give some concrete examples of definitions reuse (and maybe make it
> easier to test your methods?), I've been putting schemas mostly here,
> https://github.com/w3c/web-annotation-tests/tree/master/common. Take a
> look at bodyResource.json and targetResource.json. These schemas are
> designed to check that body/target resource(s) is(are all) instance(s) of
> classes allowed to be a body or target. Both schemas use essentially the
> same definitions (the bodyResource.json has a textualBody definition not
> used in the targetResource.json). There's also one schema I just put in a
> sibling folder /TextualBody/textualBody.json. This schema is designed to
> return valid if any body of an Annotation is a TextualBody and reuses one
> of the common definitions and adds an additional definition that will be
> used in multiple schemas. This schema tells us if the TextualBody feature
> was implemented in an annotation and determines if further schemas should
> be run to identify which sub-features of TextualBody were implemented and
> whether requirements re TextualBody and its sub-features were met.
>
>
>
> Which raises the question of folder design to handle all the schemas that
> are in the pipeline. Regarding top-level folders, i.e., siblings of
> /common, I've been reworking Rob's original features spreadsheet. I think
> we will need several siblings of /common, not only for definitions, but
> also for selected child objects of an Annotation that themselves can have a
> number of unique keys and their own child objects (e.g., SpecificResource,
> TextualBody as discussed above, etc.). I think this is best way to manage
> what is going to eventually be a relatively large number of schemas. If you
> want to suggest a different folder hierarchy approach please do – but can't
> all go in /common (which I think you've already said).
>
>
>
> For example, there are 17 keys in the model that can be used on an
> Annotation (16 at any one time, 2 are mutually exclusive). About 7 or 8
> of these keys (e.g., id, type, and life-cycle keys) can appear on other
> Annotation child objects, so relevant schemas will be re-used. In a few
> cases the constraints on a key varies according to it's parent. Since some
> keys are should or may, but then if used have must constraints, I'm mostly
> following a pattern of 1 schema to determine if the feature is present, and
> a second schema to determine if the feature was implemented correctly.
> Thus, id is a 'should' key on an Annotation, but if present id must be
> singular (array not allowed) and its value must be of format uri. So if
> id is not present the invalid message (first schema) is "Warning: The
> Annotation is not identified using the id key (Section 3.1)," but if id
> is present and not a uri, then the invalid message (2nd schema) is:
> "Error: The Annotation identifier ('id') is not a string of format uri
> (Section 3.1)." If the first schema returns invalid, the second schema
> is skipped. You get the idea. So in the /common folder I think we will end
> up with about 35 to 40 schemas. Hence the need for sibling folders I've
> added (or alternatively for child folders).
>
>
>
> I don't know as we'll get this far into the weeds on the call Friday, but
> wanted to flag folder management as a topic of discussion on email at
> least.
>
>
>
> I will post a message tomorrow with a link to an incomplete version of
> revised spread sheet and about schema granularity to get this conversation
> up and going again. Not necessarily critical for the testing plan we need
> for CR, but to actually get to point where we can start running test, we
> need to figure this stuff out.
>
>
>
> -Tim Cole
>
>
>
>
>
>
>
> *From:* Testdev [mailto:testdev-bounces@lists.spec-ops.io] *On Behalf Of *Shane
> McCarron
> *Sent:* Wednesday, June 08, 2016 4:30 PM
>
> *To:* Discussions about Test Development <testdev@lists.spec-ops.io>
> *Cc:* W3C Public Annotation List <public-annotation@w3.org>
> *Subject:* Re: [Testdev] [model-test] referencing external schema
>
>
>
> Okay. We have two strategies in mind:
>
>
>
> 1) have a top level definitions folder, then use absolute URIs (e.g.,
> /annotation-model/definitions/my_def.json)
>
> 2) have that top level folder, and have ajv automatically import all of
> the .json files in that folder during initialization.
>
>
>
> Would either of those work for you Tim?
>
>
>
>
>
> On Wed, Jun 8, 2016 at 2:50 PM, Timothy Cole <t-cole3@illinois.edu> wrote:
>
> Yes. I actually had created the definitions folder before I realized
> relative addressing wasn't working. I'm in the process of creating some
> better examples of re-useable definitions – the samples I included in my
> email are out of date and will be replaced.
>
>
>
> Thanks,
>
>
>
> Tim Cole
>
>
>
>
>
> *From:* Testdev [mailto:testdev-bounces@lists.spec-ops.io] *On Behalf Of *Benjamin
> Young
> *Sent:* Wednesday, June 08, 2016 2:26 PM
>
>
> *To:* Discussions about Test Development <testdev@lists.spec-ops.io>
> *Cc:* W3C Public Annotation List <public-annotation@w3.org>
> *Subject:* Re: [Testdev] [model-test] referencing external schema
>
>
>
> Hey Tim,
>
>
>
> Sorry for the wait. Shane and I just chatted about this $ref thing and I
> think we can make it work in WPT. It’s mostly a question of what the
> relative path expectations of `ajv` tucked inside of WPT might be.
>
>
>
> It looks like we may also be able to hack around the defaults (if need be)
> using wptserve’s Pipes system:
>
> https://wptserve.readthedocs.io/en/latest/pipes.html
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__wptserve.readthedocs.io_en_latest_pipes.html&d=CwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=CKlycpO63cn6OBnNNc-wEYw8fUsLLS2T3j_BvQUxqM4&s=yKFYr73mCj5fVvAFd-pW7tFFDalI4fVfaycO-20g66Q&e=>
>
>
>
> What we’re thinking right now is taking the JSON objects you now have in
> the `definitions` object (which you pasted below) and putting those inside
> of a `definitions` folder and then working through the relative path
> details with WPT.
>
>
>
> Would that serve as a solution for what you’re attempting here?
>
>
>
> Cheers!
>
> Benjamin
>
>
>
> *From:* Testdev [mailto:testdev-bounces@lists.spec-ops.io
> <testdev-bounces@lists.spec-ops.io>] *On Behalf Of *Shane McCarron
> *Sent:* Wednesday, June 8, 2016 10:46 AM
> *To:* Discussions about Test Development <testdev@lists.spec-ops.io>
> *Cc:* W3C Public Annotation List <public-annotation@w3.org>
> *Subject:* Re: [Testdev] [model-test] referencing external schema
>
>
>
> Did you get any traction on this Tim?
>
>
>
> On Thu, Jun 2, 2016 at 10:19 AM, Timothy Cole <t-cole3@illinois.edu>
> wrote:
>
> We are currently developing json schemas to help test for implementations
> of SpecificResource features of the data model. I've uploaded 4 of the
> schemas currently being developed to
> https://github.com/w3c/web-annotation-tests/tree/master/SpecificResource
> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_w3c_web-2Dannotation-2Dtests_tree_master_SpecificResource&d=CwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=CKlycpO63cn6OBnNNc-wEYw8fUsLLS2T3j_BvQUxqM4&s=V_xozOO0wxNGrG964fbg7j8vJDgiNPr-Zf3a3z7cUss&e=>
> (these are first drafts, not quite ready, still being debugged and
> refined). All 4 schemas share a definitions section containing 2
> sub-schemas (appended below). These are easy to reference internally where
> needed, but it would be even more convenient to reference them as external
> schemas. This should be possible using $ref, but I've had difficulties
> getting this approach to work with my local ajv instance, especially with
> regard to relative file locations.
>
>
>
> Is anyone familiar enough with how $ref works within the Web Platform
> Testing framework's implementation of ajv to advise?
>
>
>
> Below is the current way the sub-schemas are embedded in each primary
> schema. Obviously this works, but it means repeating the same sub-schemas
> in multiple files, which is obviously less efficient than if these
> sub-schemas could be referenced as individual files.
>
>
>
> Thanks,
>
> Tim Cole
>
>
>
> {
>
> "definitions": {
>
> "SpecificResource": {
>
> "type": "object",
>
> "properties": {
>
> "selector": {},
>
> "state": {},
>
> "styleClass": {},
>
> "renderedVia": {},
>
> "scope": {},
>
> "purpose": {}
>
> },
>
> "anyOf": [
>
> {"required": ["selector"]},
>
> {"required": ["state"]},
>
> {"required": ["styleClass"]},
>
> {"required": ["renderedVia"]},
>
> {"required": ["scope"]},
>
> {
>
> "required": ["purpose"],
>
> "not": {"required": ["value"]}
>
> }
>
> ]
>
> },
>
> "SpecificResType" : {
>
> "type": "object",
>
> "properties": {
>
> "type": {
>
> "oneOf": [
>
> {"type": "string",
>
> "enum": ["SpecificResource"]},
>
> {"type": "array",
>
> "items": [{ "type": "string",
>
> "enum": ["SpecificResource"] }]
>
> } ]
>
> }
>
> },
>
> "required": ["type"]
>
> }
>
> }
>
> }
>
>
> _______________________________________________
> Testdev mailing list
> Testdev@lists.spec-ops.io
> http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.spec-2Dops.io_listinfo.cgi_testdev-2Dspec-2Dops.io&d=CwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=CKlycpO63cn6OBnNNc-wEYw8fUsLLS2T3j_BvQUxqM4&s=AKojwtr16bHcWXtHtf8ZR9e_-lTF7Oan-BiiKomym3I&e=>
>
>
>
>
>
> --
>
> Shane McCarron
>
> Projects Manager, Spec-Ops
>
>
> _______________________________________________
> Testdev mailing list
> Testdev@lists.spec-ops.io
> http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io
> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.spec-2Dops.io_listinfo.cgi_testdev-2Dspec-2Dops.io&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=19fTLxSHQ7CJCtVk1Lcg9AHFHSM3jG803vzgqbR4H2E&s=YjGrRdor7B_zkXJ3Vd-wgMJitXS21c41ZVhzFla6TcY&e=>
>
>
>
>
>
> --
>
> Shane McCarron
>
> Projects Manager, Spec-Ops
>
> _______________________________________________
> Testdev mailing list
> Testdev@lists.spec-ops.io
> http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io
>
>
--
Shane McCarron
Projects Manager, Spec-Ops
Received on Thursday, 9 June 2016 15:59:47 UTC