RE: testdev repository

Thanks, Shane. New files added to https://github.com/w3c/web-annotation-tests have been showing up in the http://testdev.spec-ops.io:8080 repository as expected, which is great, but I notice that old files in the testdev repository don't get deleted when corresponding files are deleted from web-annotation-tests.  It looks like these deprecated files then show up in the https://github.com/w3c/web-platform-tests when you do a PR to push files from testdev into production. 
 
Let me know what easiest way to help resolve. Details:
 
I've added the tests and assertions for annotation bodies and targets (5 tests involving about 100 assertions), but before you prepare and submit next PR to move these on up to web-platform-tests, I think it would be a good idea to get rid of the deprecated files, which are now quite numerous.  For example, there should be only 6 json files in the /definitions directory:
 
annotations.json
bodyTarget.json
choiceSet.json
id.json
otherproperties.json
specificResource.json
 
Currently the testdev repo (and web-platform-tests repo) have about 20 files in /definitions.  Since all 20 of these may be getting loaded every time AJV is being called (I think you're loading /definitions/*.json) , this is not good.
 
Similar (proportionally) issue in both the /annotations folder and the /bodiesTargets folder, especially the latter because of some changes I made over the weekend and on Monday in granularity of assertions to better align with the WG call last Friday.
 
I can (I think) submit a PR against the testdev repo proposing the files to delete, but didn't know if this is what you want me to do, or if you have another way to go to deal with deleting files no longer being used. 
 
Anyway, once this issue is resolved we should probably do another PR to add body and target tests to web-platform-tests as soon as possible. I have a few more tests to add pertaining to Specific Resources and Collections/Pages, but most implementers will just be interested in the first 8 tests already up on testdev.  Unfortunately, before doing this PR, you'll probably still need to re-run your normalization process to deal with whitespace characters that web-platform-test lint doesn't like, including on a few files from first PR that I had to modify…
 
Tonight (or more likely tomorrow morning) I'll post more about the body and target tests I've added this week, about a few suggested tweaks you might consider to the test harness (if easy), about changes we've made in model testing to better align with test/assertion granularity discussion we had last Friday, and about issues with stack space on testdev that I ran into over the weekend.  
 
Janina will begin testing with our small emblematica annotation client tomorrow and we'll exercise the report generation pipeline when we do that, so may have more feedback for you by Friday. 
 
Thanks,
 
Tim Cole
 
 
From: Shane McCarron [mailto:shane@spec-ops.io] 
Sent: Friday, August 26, 2016 11:28 AM
To: W3C Public Annotation List <public-annotation@w3.org>
Subject: testdev repository
 
I fixed the auto population of testdev from web-annotation-tests.  Thanks for your patience.

 
-- 
Shane McCarron
Projects Manager, Spec-Ops