Re: VCWG Test Suites

Manu, thanks for taking the time to write this up.

I'm interested in working on a test suite for securing W3C Verifiable
Credentials with JWTs.

I don't think there should be any HTTP requirements for such a test suite,
and I think docker & github pages should be all that is needed.

We have been maintaining this test suite at DIF:
https://github.com/decentralized-identity/JWS-Test-Suite

It would be nice to clean it up a bit, which I am happy to take point on.

I propose a new repo for the vc-jwt test suite: vc-jwt-test-suite, I would
like to be a CODEOWNER, an empty repo will be fine to start.

Regarding the core data model test suite, I don't believe HTTP should be a
requirement of that test suite either.

I don't plan to contribute substantially to the core data model test suite.

I do plan to help with a test suite for vc-di-bbs, and I think that test
suite should be independent of the other test suites given the unique
challenges of JSON-LD framing and selective disclosure.

It would be good to start gathering contributors for all test suites.

Regards,

OS




On Sat, Apr 8, 2023 at 3:05 PM Manu Sporny <msporny@digitalbazaar.com>
wrote:

> Hi all,
>
> As we march towards the Candidate Recommendation phase, it's important
> for us to have a believable test suite story. The Test Suite we used
> in VC 1.0 and 1.1 left much to be desired, a few of those things
> being:
>
> * The v1.x test suite was only run a handful of times by implementers
> and then not used thereafter. This resulted in implementations
> drifting in their conformance over time. If we had to do this again,
> we'd adopt a continuous integration / testing approach where the test
> suite would be run against all current implementations on a regular
> basis.
>
> * It didn't test the "securing" mechanism for VCs, and focused on just
> the normative data model statements. We need to test the securing
> mechanisms this time around.
>
> * It required implementers to submit test reports manually, even if
> they had an automated process for generating the test reports.
>
> Fast forward a few years and many of us have participated in plugfests
> that provided better interoperability testing infrastructure that
> overcame all of the shortcomings listed above. For example, the recent
> Jobs for the Future plugfest demonstrated interoperability between 17
> different issuers with an updated test suite infrastructure (see
> slides 4 and 5):
>
>
> https://docs.google.com/presentation/d/19GmJ3bLMrbVadesnkmsWaaUr-U71Y9Kr775tZvgs-xI/edit
>
> Thanks to Andrew Jones and Charles Lehner, we now have a VC Data Model
> 2.0 test suite that tests 22 of the 44 normative statements in the VC
> Data Model 2.0 specification (see example conformance report attached
> to this email):
>
> https://github.com/digitalbazaar/vc-data-model-2-test-suite
>
> We also have test suites that can test the VC Data Integrity base
> specification as well as each cryptosuite specification:
>
> https://w3c-ccg.github.io/di-ed25519-test-suite/
>
> While most of the implementations today integrate using the VC API,
> the test suites are flexible enough that other "drivers" could be
> added to support Docker-based workflows as well. These need not be the
> only test suites that the WG employs, if others want to contribute
> test suites, I imagine those would be welcomed discussion.
>
> VCWG Chairs, this is a request for some call time (main call or
> special topic call) to consider the adoption of at least the
> vc-data-model-2-test-suite and the di-ed25519-test-suite listed above.
>
> -- manu
>
> --
> Manu Sporny - https://www.linkedin.com/in/manusporny/
> Founder/CEO - Digital Bazaar, Inc.
> News: Digital Bazaar Announces New Case Studies (2021)
> https://www.digitalbazaar.com/
>


-- 
*ORIE STEELE*
Chief Technical Officer
www.transmute.industries

<https://www.transmute.industries>