Re: Testing progress

Thanks Shane, Gregg!

All of the examples in the specs are separately available via github pages,
but not obviously in the main W3C site.

A collection of the examples:

http://w3c.github.io/web-annotation/model/wd2/examples/correct/collection1.json

Rob



On Thu, Apr 21, 2016 at 12:59 PM, Shane McCarron <shane@spec-ops.io> wrote:

> We have had a lot of discussion in the past couple of weeks about how to
> do testing.  I feel that we have a clear path forward that will permit
> exercising the data model and the protocol.  There are a lot of open
> questions, but we are making progress.
>
> Specifically, I have established a "Project" within Spec-Ops around test
> development, and made Web Annotation a task of that Project.  This is
> mostly internal nonsense, and you all don't really care.
>
> I set up a discussion list within Spec-Ops for the Project that anyone can
> join [1].  There is also a wiki [2] where we will be  assembling concepts
> as they develop.  I am doing this in Spec-Ops space just because it is
> expedient and because this work applies across several projects that we are
> or plan to be working on in the next couple of months.
>
> Spec-Ops has also forked the Web Platform Tests (WPT) GitHub repo [3] so
> that there is a place for us to put the tests and associated tools.  The
> concept here is that there is a familiar environment in which to develop
> the tests and tools, and of course to run the tests and capture results.
>
> In a nutshell, we are looking at writing tests using a declarative grammar
> that will be familiar to most of you since it looks just like the JSON you
> have been writing in your spec.  A tool on the test server will parse the
> test files and present information about the tests through the WPT
> interface.  Test data will be analyzed (in the user agent or on the server)
> and a result recorded.
>
> This general model can be used not just for Web Annotation, but also for
> other specifications that rely only JSON message passing (there are lots of
> these).  But rest assured, we are not going sit around trying to craft the
> perfect general case solution.  That's not how I like to operate.  Gregg
> Kellogg (cc'd on this) has a lot of experience with this sort of testing,
> and I am confident that we can leverage his knowledge to get something in
> place quickly.
>
> More on the call tomorrow.
>
> [1] http://lists.spec-ops.io/listinfo.cgi/testdev-spec-ops.io
> [2] https://wiki.spec-ops.io/wiki/AnnotationTesting
> [3] https://github.com/Spec-Ops/web-platform-tests
>
> P.S. It is very hard to write tests without implementations to test
> against.  We can (and will) mock up some golden data to test the
> infrastructure... but it would be great to have access to implementations
> sooner than later.  Even buggy ones!
>
> --
> Shane McCarron
> Projects Manager, Spec-Ops
>



-- 
Rob Sanderson
Information Standards Advocate
Digital Library Systems and Services
Stanford, CA 94305