- From: Shane McCarron <shane@spec-ops.io>
- Date: Sat, 11 Jun 2016 11:23:06 -0500
- To: Discussions about Test Development <testdev@lists.spec-ops.io>
- Cc: W3C Public Annotation List <public-annotation@w3.org>
- Message-ID: <CAJdbnODUXkBwqdXEpwSXQUXWjRhQo76hbMN_ti-mffE_ZjV61Q@mail.gmail.com>
I am going to add these top level folders today so that I can continue to
make progress. This may require moving some things around. I hope that
doesn't disrupt anything too much.
On Thu, Jun 9, 2016 at 12:44 PM, Shane McCarron <shane@spec-ops.io> wrote:
> Typically the section numbers are not included in the directory name. So
> I would do something like:
>
> /annotations
> /bodiesTargets
> /specificResources
> /specificResources/selectors
> /specificResources/states
> /collections
>
> In other words, structure it like the hierarchy of the spec itself. Does
> that make sense? And yes, go ahead and add them.
>
> And if you think things are better in subfolders than in common, go for
> that. I assumed things would get used a lot and so we would need a
> "common" folder. If that is NOT the case, should we change the rule on
> relative "assertionFile" paths to be relative to the current directory
> rather than to test suite top?
>
>
> On Thu, Jun 9, 2016 at 11:47 AM, Timothy Cole <t-cole3@illinois.edu>
> wrote:
>
>> Okay, so obvious folders for the annotation data model would be:
>>
>>
>>
>> /common
>>
>> /definitions (as we've discussed)
>>
>> /3.1annotations
>>
>> /3.2bodiesTargets
>>
>> /4.specificResources
>>
>> /4.2selectors
>>
>> /4.3states
>>
>> /5.collections
>>
>>
>>
>> We can leave the numbers off if preferred, but obviously these tie to
>> specific sections of the model spec. And of course happy to follow wpt
>> preferred naming patterns if you let us know what those are.
>>
>>
>>
>> Everything currently in /common would need to be redistributed to
>> 3.1annotations or 3.2bodiesTargets; almost none of the schema currently in
>> this folder (with possibly exception of id.json and idFormat.json) would
>> be re-used for checking implementation of anything other than the
>> Annotation object itself, so unlikely to appear in multiple tests.
>>
>>
>>
>> Note also that to avoid making too many folders I'm suggesting that the
>> 4.specificResources folder contain schema for all parts of Section 4 not
>> specific to selectors and states (i.e., would include schema for checking
>> and validating implementation of purpose, styles, renderedVia and scope
>> as well as for identifying and checking implementations of
>> SpecificResource generally).
>>
>>
>>
>> Could also add a 3.3otherProperties folder, but this folder would contain
>> schema that would be used across multiple sections, so would it be better
>> to use /common for these schema or to create the additional folder?
>>
>>
>>
>> Does this folder list make sense in context of wpt best practices?
>>
>>
>>
>> Let me know preferred naming convention, and whether okay to add these
>> half-dozen folders as siblings to /common.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Tim Cole
>>
>>
>>
>>
>>
>>
>>
>> *From:* Testdev [mailto:testdev-bounces@lists.spec-ops.io] *On Behalf Of
>> *Shane McCarron
>> *Sent:* Thursday, June 09, 2016 10:59 AM
>> *To:* Discussions about Test Development <testdev@lists.spec-ops.io>
>> *Cc:* W3C Public Annotation List <public-annotation@w3.org>
>> *Subject:* Re: [Testdev] [model-test] referencing external schema
>>
>>
>>
>> Typically WPT folder trees are designed to mirror the structure of the
>> spec being tested. That way it is easy to correlate the assertions with
>> the content of the spec. Beyond this, there are sometimes top level
>> folders like "tools" or "common". I think it is important that we stick
>> with that sort of structure to help with adoption by the wpt community.
>>
>>
>>
>> On Wed, Jun 8, 2016 at 5:37 PM, Timothy Cole <t-cole3@illinois.edu>
>> wrote:
>>
>> I think either approach should work, but I've had some issues even with
>> referencing absolute URIs . The ajv import approach might be a little
>> easier and more reliable? Regardless, now that the patterns are becoming
>> clear, copying definitions to multiple schemas not all that onerous. You
>> can always include definitions in a schema and not reference… Obviously
>> the big issue is when you want to revise a definition (or upgrade to schema
>> version 5). Then having to go back and fix in multiple files would indeed
>> be an issue. So hopefully one of your approaches will work.
>>
>>
>>
>> To give some concrete examples of definitions reuse (and maybe make it
>> easier to test your methods?), I've been putting schemas mostly here,
>> https://github.com/w3c/web-annotation-tests/tree/master/common
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_w3c_web-2Dannotation-2Dtests_tree_master_common&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=PAgr-EMmzxbTn6qrxoI4TsfCVKa_y2CmyTu94a2OWgk&s=w3t92nO3Tnf4MBgucqxYOoQx7AAT87SEOTz7vWzHCmU&e=>.
>> Take a look at bodyResource.json and targetResource.json. These schemas are
>> designed to check that body/target resource(s) is(are all) instance(s) of
>> classes allowed to be a body or target. Both schemas use essentially the
>> same definitions (the bodyResource.json has a textualBody definition not
>> used in the targetResource.json). There's also one schema I just put in a
>> sibling folder /TextualBody/textualBody.json. This schema is designed to
>> return valid if any body of an Annotation is a TextualBody and reuses one
>> of the common definitions and adds an additional definition that will be
>> used in multiple schemas. This schema tells us if the TextualBody feature
>> was implemented in an annotation and determines if further schemas should
>> be run to identify which sub-features of TextualBody were implemented and
>> whether requirements re TextualBody and its sub-features were met.
>>
>>
>>
>> Which raises the question of folder design to handle all the schemas that
>> are in the pipeline. Regarding top-level folders, i.e., siblings of
>> /common, I've been reworking Rob's original features spreadsheet. I think
>> we will need several siblings of /common, not only for definitions, but
>> also for selected child objects of an Annotation that themselves can have a
>> number of unique keys and their own child objects (e.g., SpecificResource,
>> TextualBody as discussed above, etc.). I think this is best way to manage
>> what is going to eventually be a relatively large number of schemas. If you
>> want to suggest a different folder hierarchy approach please do – but can't
>> all go in /common (which I think you've already said).
>>
>>
>>
>> For example, there are 17 keys in the model that can be used on an
>> Annotation (16 at any one time, 2 are mutually exclusive). About 7 or 8 of
>> these keys (e.g., id, type, and life-cycle keys) can appear on other
>> Annotation child objects, so relevant schemas will be re-used. In a few
>> cases the constraints on a key varies according to it's parent. Since some
>> keys are should or may, but then if used have must constraints, I'm mostly
>> following a pattern of 1 schema to determine if the feature is present, and
>> a second schema to determine if the feature was implemented correctly.
>> Thus, id is a 'should' key on an Annotation, but if present id must be
>> singular (array not allowed) and its value must be of format uri. So if id
>> is not present the invalid message (first schema) is "Warning: The
>> Annotation is not identified using the id key (Section 3.1)," but if id is
>> present and not a uri, then the invalid message (2nd schema) is: "Error:
>> The Annotation identifier ('id') is not a string of format uri (Section
>> 3.1)." If the first schema returns invalid, the second schema is skipped.
>> You get the idea. So in the /common folder I think we will end up with
>> about 35 to 40 schemas. Hence the need for sibling folders I've added (or
>> alternatively for child folders).
>>
>>
>>
>> I don't know as we'll get this far into the weeds on the call Friday, but
>> wanted to flag folder management as a topic of discussion on email at
>> least.
>>
>>
>>
>> I will post a message tomorrow with a link to an incomplete version of
>> revised spread sheet and about schema granularity to get this conversation
>> up and going again. Not necessarily critical for the testing plan we need
>> for CR, but to actually get to point where we can start running test, we
>> need to figure this stuff out.
>>
>>
>>
>> -Tim Cole
>>
>>
>>
>>
>>
>>
>>
>> *From:* Testdev [mailto:testdev-bounces@lists.spec-ops.io] *On Behalf Of
>> *Shane McCarron
>> *Sent:* Wednesday, June 08, 2016 4:30 PM
>>
>>
>> *To:* Discussions about Test Development <testdev@lists.spec-ops.io>
>> *Cc:* W3C Public Annotation List <public-annotation@w3.org>
>> *Subject:* Re: [Testdev] [model-test] referencing external schema
>>
>>
>>
>> Okay. We have two strategies in mind:
>>
>>
>>
>> 1) have a top level definitions folder, then use absolute URIs (e.g.,
>> /annotation-model/definitions/my_def.json)
>>
>> 2) have that top level folder, and have ajv automatically import all of
>> the .json files in that folder during initialization.
>>
>>
>>
>> Would either of those work for you Tim?
>>
>>
>>
>>
>>
>> On Wed, Jun 8, 2016 at 2:50 PM, Timothy Cole <t-cole3@illinois.edu>
>> wrote:
>>
>> Yes. I actually had created the definitions folder before I realized
>> relative addressing wasn't working. I'm in the process of creating some
>> better examples of re-useable definitions – the samples I included in my
>> email are out of date and will be replaced.
>>
>>
>>
>> Thanks,
>>
>>
>>
>> Tim Cole
>>
>>
>>
>>
>>
>> *From:* Testdev [mailto:testdev-bounces@lists.spec-ops.io] *On Behalf Of
>> *Benjamin Young
>> *Sent:* Wednesday, June 08, 2016 2:26 PM
>>
>>
>> *To:* Discussions about Test Development <testdev@lists.spec-ops.io>
>> *Cc:* W3C Public Annotation List <public-annotation@w3.org>
>> *Subject:* Re: [Testdev] [model-test] referencing external schema
>>
>>
>>
>> Hey Tim,
>>
>>
>>
>> Sorry for the wait. Shane and I just chatted about this $ref thing and I
>> think we can make it work in WPT. It’s mostly a question of what the
>> relative path expectations of `ajv` tucked inside of WPT might be.
>>
>>
>>
>> It looks like we may also be able to hack around the defaults (if need
>> be) using wptserve’s Pipes system:
>>
>> https://wptserve.readthedocs.io/en/latest/pipes.html
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__wptserve.readthedocs.io_en_latest_pipes.html&d=CwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=CKlycpO63cn6OBnNNc-wEYw8fUsLLS2T3j_BvQUxqM4&s=yKFYr73mCj5fVvAFd-pW7tFFDalI4fVfaycO-20g66Q&e=>
>>
>>
>>
>> What we’re thinking right now is taking the JSON objects you now have in
>> the `definitions` object (which you pasted below) and putting those inside
>> of a `definitions` folder and then working through the relative path
>> details with WPT.
>>
>>
>>
>> Would that serve as a solution for what you’re attempting here?
>>
>>
>>
>> Cheers!
>>
>> Benjamin
>>
>>
>>
>> *From:* Testdev [mailto:testdev-bounces@lists.spec-ops.io
>> <testdev-bounces@lists.spec-ops.io>] *On Behalf Of *Shane McCarron
>> *Sent:* Wednesday, June 8, 2016 10:46 AM
>> *To:* Discussions about Test Development <testdev@lists.spec-ops.io>
>> *Cc:* W3C Public Annotation List <public-annotation@w3.org>
>> *Subject:* Re: [Testdev] [model-test] referencing external schema
>>
>>
>>
>> Did you get any traction on this Tim?
>>
>>
>>
>> On Thu, Jun 2, 2016 at 10:19 AM, Timothy Cole <t-cole3@illinois.edu>
>> wrote:
>>
>> We are currently developing json schemas to help test for implementations
>> of SpecificResource features of the data model. I've uploaded 4 of the
>> schemas currently being developed to
>> https://github.com/w3c/web-annotation-tests/tree/master/SpecificResource
>> <https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_w3c_web-2Dannotation-2Dtests_tree_master_SpecificResource&d=CwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=CKlycpO63cn6OBnNNc-wEYw8fUsLLS2T3j_BvQUxqM4&s=V_xozOO0wxNGrG964fbg7j8vJDgiNPr-Zf3a3z7cUss&e=>
>> (these are first drafts, not quite ready, still being debugged and
>> refined). All 4 schemas share a definitions section containing 2
>> sub-schemas (appended below). These are easy to reference internally where
>> needed, but it would be even more convenient to reference them as external
>> schemas. This should be possible using $ref, but I've had difficulties
>> getting this approach to work with my local ajv instance, especially with
>> regard to relative file locations.
>>
>>
>>
>> Is anyone familiar enough with how $ref works within the Web Platform
>> Testing framework's implementation of ajv to advise?
>>
>>
>>
>> Below is the current way the sub-schemas are embedded in each primary
>> schema. Obviously this works, but it means repeating the same sub-schemas
>> in multiple files, which is obviously less efficient than if these
>> sub-schemas could be referenced as individual files.
>>
>>
>>
>> Thanks,
>>
>> Tim Cole
>>
>>
>>
>> {
>>
>> "definitions": {
>>
>> "SpecificResource": {
>>
>> "type": "object",
>>
>> "properties": {
>>
>> "selector": {},
>>
>> "state": {},
>>
>> "styleClass": {},
>>
>> "renderedVia": {},
>>
>> "scope": {},
>>
>> "purpose": {}
>>
>> },
>>
>> "anyOf": [
>>
>> {"required": ["selector"]},
>>
>> {"required": ["state"]},
>>
>> {"required": ["styleClass"]},
>>
>> {"required": ["renderedVia"]},
>>
>> {"required": ["scope"]},
>>
>> {
>>
>> "required": ["purpose"],
>>
>> "not": {"required": ["value"]}
>>
>> }
>>
>> ]
>>
>> },
>>
>> "SpecificResType" : {
>>
>> "type": "object",
>>
>> "properties": {
>>
>> "type": {
>>
>> "oneOf": [
>>
>> {"type": "string",
>>
>> "enum": ["SpecificResource"]},
>>
>> {"type": "array",
>>
>> "items": [{ "type": "string",
>>
>> "enum": ["SpecificResource"] }]
>>
>> } ]
>>
>> }
>>
>> },
>>
>> "required": ["type"]
>>
>> }
>>
>> }
>>
>> }
>>
>>
>> _______________________________________________
>> Testdev mailing list
>> Testdev@lists.spec-ops.io
>> http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.spec-2Dops.io_listinfo.cgi_testdev-2Dspec-2Dops.io&d=CwMGaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=CKlycpO63cn6OBnNNc-wEYw8fUsLLS2T3j_BvQUxqM4&s=AKojwtr16bHcWXtHtf8ZR9e_-lTF7Oan-BiiKomym3I&e=>
>>
>>
>>
>>
>>
>> --
>>
>> Shane McCarron
>>
>> Projects Manager, Spec-Ops
>>
>>
>> _______________________________________________
>> Testdev mailing list
>> Testdev@lists.spec-ops.io
>> http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.spec-2Dops.io_listinfo.cgi_testdev-2Dspec-2Dops.io&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=19fTLxSHQ7CJCtVk1Lcg9AHFHSM3jG803vzgqbR4H2E&s=YjGrRdor7B_zkXJ3Vd-wgMJitXS21c41ZVhzFla6TcY&e=>
>>
>>
>>
>>
>>
>> --
>>
>> Shane McCarron
>>
>> Projects Manager, Spec-Ops
>>
>>
>> _______________________________________________
>> Testdev mailing list
>> Testdev@lists.spec-ops.io
>> http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io
>> <https://urldefense.proofpoint.com/v2/url?u=http-3A__lists.spec-2Dops.io_listinfo.cgi_testdev-2Dspec-2Dops.io&d=CwMFaQ&c=8hUWFZcy2Z-Za5rBPlktOQ&r=zjI0r-H6xRs5fYf2_jJkju6US9ijk0nLw4ns2nuwU2k&m=PAgr-EMmzxbTn6qrxoI4TsfCVKa_y2CmyTu94a2OWgk&s=_kDSQVBXeFOn5Tus9eXuhQFsmzNjhnYsnTQvN0xcGVM&e=>
>>
>>
>>
>>
>>
>> --
>>
>> Shane McCarron
>>
>> Projects Manager, Spec-Ops
>>
>> _______________________________________________
>> Testdev mailing list
>> Testdev@lists.spec-ops.io
>> http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io
>>
>>
>
>
> --
> Shane McCarron
> Projects Manager, Spec-Ops
>
--
Shane McCarron
Projects Manager, Spec-Ops
Received on Saturday, 11 June 2016 16:24:04 UTC