Re: Testing progress

Oh, and just to be clear, it is NOT just Shane and Gregg.  Current players
include:

Gregg Kellogg
Randall Leeds
Dave Longley
Shane McCarron
Doug Schepers
Benjamin Young

And, as I said, this is part of a larger effort so expect other players
sooner than later.  Thanks for inviting us to help address your problem.
It has been a catalyst!


On Thu, Apr 21, 2016 at 6:53 PM, Shane McCarron <shane@spec-ops.io> wrote:

> That's fantastic.  Thanks!
>
> On Thu, Apr 21, 2016 at 3:12 PM, Robert Sanderson <azaroth42@gmail.com>
> wrote:
>
>>
>> Thanks Shane, Gregg!
>>
>> All of the examples in the specs are separately available via github
>> pages, but not obviously in the main W3C site.
>>
>> A collection of the examples:
>>
>> http://w3c.github.io/web-annotation/model/wd2/examples/correct/collection1.json
>>
>> Rob
>>
>>
>>
>> On Thu, Apr 21, 2016 at 12:59 PM, Shane McCarron <shane@spec-ops.io>
>> wrote:
>>
>>> We have had a lot of discussion in the past couple of weeks about how to
>>> do testing.  I feel that we have a clear path forward that will permit
>>> exercising the data model and the protocol.  There are a lot of open
>>> questions, but we are making progress.
>>>
>>> Specifically, I have established a "Project" within Spec-Ops around test
>>> development, and made Web Annotation a task of that Project.  This is
>>> mostly internal nonsense, and you all don't really care.
>>>
>>> I set up a discussion list within Spec-Ops for the Project that anyone
>>> can join [1].  There is also a wiki [2] where we will be  assembling
>>> concepts as they develop.  I am doing this in Spec-Ops space just because
>>> it is expedient and because this work applies across several projects that
>>> we are or plan to be working on in the next couple of months.
>>>
>>> Spec-Ops has also forked the Web Platform Tests (WPT) GitHub repo [3] so
>>> that there is a place for us to put the tests and associated tools.  The
>>> concept here is that there is a familiar environment in which to develop
>>> the tests and tools, and of course to run the tests and capture results.
>>>
>>> In a nutshell, we are looking at writing tests using a declarative
>>> grammar that will be familiar to most of you since it looks just like the
>>> JSON you have been writing in your spec.  A tool on the test server will
>>> parse the test files and present information about the tests through the
>>> WPT interface.  Test data will be analyzed (in the user agent or on the
>>> server) and a result recorded.
>>>
>>> This general model can be used not just for Web Annotation, but also for
>>> other specifications that rely only JSON message passing (there are lots of
>>> these).  But rest assured, we are not going sit around trying to craft the
>>> perfect general case solution.  That's not how I like to operate.  Gregg
>>> Kellogg (cc'd on this) has a lot of experience with this sort of testing,
>>> and I am confident that we can leverage his knowledge to get something in
>>> place quickly.
>>>
>>> More on the call tomorrow.
>>>
>>> [1] http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io
>>> [2] https://wiki.spec-ops.io/wiki/AnnotationTesting
>>> [3] https://github.com/Spec-Ops/web-platform-tests
>>>
>>> P.S. It is very hard to write tests without implementations to test
>>> against.  We can (and will) mock up some golden data to test the
>>> infrastructure... but it would be great to have access to implementations
>>> sooner than later.  Even buggy ones!
>>>
>>> --
>>> Shane McCarron
>>> Projects Manager, Spec-Ops
>>>
>>
>>
>>
>> --
>> Rob Sanderson
>> Information Standards Advocate
>> Digital Library Systems and Services
>> Stanford, CA 94305
>>
>
>
>
> --
> Shane McCarron
> Projects Manager, Spec-Ops
>



-- 
Shane McCarron
Projects Manager, Spec-Ops