Re: Test reports and interop (was Re: We've moved on from proposing the "Solid" WG (PUMPKIN WG))

The work you refer to is still ongoing. There are over 600 tests but
coverage is still not complete - we were hoping for more community
contributions to this.
* There is a QA spec defining how we manage and report on tests:
* The link you were looking at is for results from one of the test suites,
the other test suite is here:
* This produces a coverage report which includes the protocol spec and
access control elements common to both the WAC and ACP specs:

On Mon, 25 Mar 2024 at 12:38, Kjetil Kjernsmo <> wrote:

> On mandag 25. mars 2024 11:16:21 CET Sjoerd van Groning wrote:
> > > Got reference to authoritative/verifiable/reproducible test reports?
> >
> >
> I have to admit, this is not where I hoped Solid would have gone. Back in
> the
> day, the vision was to have anchors in the specs for every requirement, so
> that interested parties would have a full coverage report, so that it
> would be
> easy to verify that every requirement had at least a test, and so that it
> would be easy to see what requirement a test belonged to. In addition,
> there
> would be a test suite that tested things that weren't in the spec, like
> interpretations.
> Just having a test suite that ticks a box when all tests pass does not
> have
> much to offer for people wishing to see evidence of actual interop before
> they
> dive in, then, they would have to first write their app, and then see if
> it
> works with all servers. I wouldn't have made that investment to be frank.
> I would like to see coverage reports, what parts of the spec are covered
> and
> what are not covered, I would like to see linked a full test report where
> each
> test is explained for every server. That would be actually transparent.
> And I note that we did have this more or less implemented for the protocol
> spec. Coverage wasn't great, and things were so much in flux back then
> that it
> was too early to publish full reports, but the functionality was more or
> less
> there.
> Kind regards,
> Kjetil

This e-mail, and any attachments thereto, is intended only for use by the 
addressee(s) named herein and may contain legally privileged, confidential 
and/or proprietary information. If you are not the intended recipient of 
this e-mail (or the person responsible for delivering this document to the 
intended recipient), please do not disseminate, distribute, print or copy 
this e-mail, or any attachment thereto. If you have received this e-mail in 
error, please respond to the individual sending the message, and 
permanently delete the email.

Received on Monday, 25 March 2024 12:54:47 UTC