Re: Solid QA implementations

Just to add some detail to what Michiel has said, the
https://github.com/solid-contrib/specification-tests repo contains over 600
tests for the Solid Protocol and WAC/ACP specs. Many requirements in the
specs are linked to groups of tests via the test manifests such as htt
ps://github.com/solid-contrib/specification-tests/blob/main/protocol/solid-protocol-test-manifest.ttl
and we define test subjects in RDF using
https://github.com/solid-contrib/specification-tests/blob/main/test-subjects.ttl.
The tests are run by the Conformance Test Harness (
https://github.com/solid-contrib/conformance-test-harness). This loads the
manifests and specifications into an RDF model in order to drive the tests.
As well as running tests and producing reports which are reasonably inline
with the Solid QA spec, it can produce a coverage report (
https://solid-contrib.github.io/specification-tests/coverage) showing the
relationship between specs, manifests and test features. It was very
satisfying to be able to drive all these tests and reports from LinkedData
but there is more work to be done, for example, adding annotations to
manifests and reports; and aligning better with the Solid QA work.

Pete

On Fri, 9 Feb 2024 at 08:52, Michiel de Jong <michiel@unhosted.org> wrote:

> Hi Benjamin,
>
> Thanks for the shout-out! That document was mainly Sarven's work, Pete and
> I and others helped review it.
> Did you see https://github.com/solid-contrib/specification-tests? That is
> mostly Pete's work, great if you want to reuse it!
> Most of our tests are still only linked to the spec through imprecise code
> comments and general intuition, but as the doc states
> we aim to migrate all that and make it all machine-readable using RDF. The
> establishment of a Solid WG at the W3C level could
> definitely help us to push that migration effort forward, and it would
> indeed be great to share lessons learned with you and your
> work on AP testing! Let us know if you have anything to show off on your
> side.
>
> You can come to our test suite matrix channel (
> https://matrix.to/#/#solid_test-suite:gitter.im) if you need help with
> anything.
>
> Cheers,
> Michiel
>
> On Wed, 7 Feb 2024 at 04:23, Benjamin Goering <ben@bengo.co> wrote:
>
>> Hey there. I just wanted to shout and say that I found the document at
>> https://solidproject.org/ED/qa really interesting when considering how
>> to represent test cases for ActivityPub
>> https://www.w3.org/TR/activitypub/
>>
>> It seems[0] to jive well with the rest of this test methodology, at least
>> at a high level https://www.w3.org/TR/test-methodology/
>>
>> In activitypub-testing[1], a test runner for activitypub, I’d like to
>> explore ways of publishing test cases that interop with and/or share
>> tooling with the “Test Case Descriptions” from Solid QA
>> https://solidproject.org/ED/qa#test-case-description
>> That way the tool could support browsing and maybe even running tests
>> published anywhere on the web as long as it can fetch/parse the Test Case
>> Description or Test Suite by URL.
>>
>> I’m collecting a list of known implementations of Test Case Description
>> from Solid QA so I could use them as test fixtures in any fetcher/parser.
>> Anyone wanna show off an example test case description or two that I aim to
>> emulate?
>>
>> I want to do something similar for to describe test cases like
>> https://socialweb.coop/activitypub/test-cases/outbox-post-servers-must-return-a-201-created-http-code/
>> or
>> https://activitypub-testing-website.socialweb.coop/test-cases/actor-must-serve-as2-object-to-get
>> For example, I could try to modify the html to mark up some of the
>> properties of the test case description using rdfa and/or I could also
>> embed a <script> tag with JSON-LD (or any other rdf) with the data all in
>> one place. I could also work on content negotiation for various media types
>> at the http layer. Is everyone already doing the same thing?
>>
>> Thanks again to the authors of that Solid QA doc. Good work so far!
>>
>> [0]: https://bengo.is/activitypub/apply-w3-test-methodology/
>> [1]: https://socialweb.coop/activitypub-testing/
>>
>>
>>

-- 
This e-mail, and any attachments thereto, is intended only for use by the 
addressee(s) named herein and may contain legally privileged, confidential 
and/or proprietary information. If you are not the intended recipient of 
this e-mail (or the person responsible for delivering this document to the 
intended recipient), please do not disseminate, distribute, print or copy 
this e-mail, or any attachment thereto. If you have received this e-mail in 
error, please respond to the individual sending the message, and 
permanently delete the email.