Re: Testing progress

That's fantastic.  Thanks!

On Thu, Apr 21, 2016 at 3:12 PM, Robert Sanderson <azaroth42@gmail.com>
wrote:

>
> Thanks Shane, Gregg!
>
> All of the examples in the specs are separately available via github
> pages, but not obviously in the main W3C site.
>
> A collection of the examples:
>
> http://w3c.github.io/web-annotation/model/wd2/examples/correct/collection1.json
>
> Rob
>
>
>
> On Thu, Apr 21, 2016 at 12:59 PM, Shane McCarron <shane@spec-ops.io>
> wrote:
>
>> We have had a lot of discussion in the past couple of weeks about how to
>> do testing.  I feel that we have a clear path forward that will permit
>> exercising the data model and the protocol.  There are a lot of open
>> questions, but we are making progress.
>>
>> Specifically, I have established a "Project" within Spec-Ops around test
>> development, and made Web Annotation a task of that Project.  This is
>> mostly internal nonsense, and you all don't really care.
>>
>> I set up a discussion list within Spec-Ops for the Project that anyone
>> can join [1].  There is also a wiki [2] where we will be  assembling
>> concepts as they develop.  I am doing this in Spec-Ops space just because
>> it is expedient and because this work applies across several projects that
>> we are or plan to be working on in the next couple of months.
>>
>> Spec-Ops has also forked the Web Platform Tests (WPT) GitHub repo [3] so
>> that there is a place for us to put the tests and associated tools.  The
>> concept here is that there is a familiar environment in which to develop
>> the tests and tools, and of course to run the tests and capture results.
>>
>> In a nutshell, we are looking at writing tests using a declarative
>> grammar that will be familiar to most of you since it looks just like the
>> JSON you have been writing in your spec.  A tool on the test server will
>> parse the test files and present information about the tests through the
>> WPT interface.  Test data will be analyzed (in the user agent or on the
>> server) and a result recorded.
>>
>> This general model can be used not just for Web Annotation, but also for
>> other specifications that rely only JSON message passing (there are lots of
>> these).  But rest assured, we are not going sit around trying to craft the
>> perfect general case solution.  That's not how I like to operate.  Gregg
>> Kellogg (cc'd on this) has a lot of experience with this sort of testing,
>> and I am confident that we can leverage his knowledge to get something in
>> place quickly.
>>
>> More on the call tomorrow.
>>
>> [1] http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io
>> [2] https://wiki.spec-ops.io/wiki/AnnotationTesting
>> [3] https://github.com/Spec-Ops/web-platform-tests
>>
>> P.S. It is very hard to write tests without implementations to test
>> against.  We can (and will) mock up some golden data to test the
>> infrastructure... but it would be great to have access to implementations
>> sooner than later.  Even buggy ones!
>>
>> --
>> Shane McCarron
>> Projects Manager, Spec-Ops
>>
>
>
>
> --
> Rob Sanderson
> Information Standards Advocate
> Digital Library Systems and Services
> Stanford, CA 94305
>



-- 
Shane McCarron
Projects Manager, Spec-Ops