Re: VCWG Test Suites

Orie,
Thank you for volunteering to clean that test suite up - it is very clean
and we can work with that.

I too would prefer a simple docker approach where implementers provide an
app in docker that takes a test input and provides an output and that does
not require any network or other connections.

I would prefer to have a separate test set for VC-JWT, and we will
contribute an independent implementation for all items covered under that
scenario.

Mike Prorock
CTO, Founder
https://mesur.io/



On Thu, Apr 13, 2023 at 8:32 AM Orie Steele <orie@transmute.industries>
wrote:

> Manu, thanks for taking the time to write this up.
>
> I'm interested in working on a test suite for securing W3C Verifiable
> Credentials with JWTs.
>
> I don't think there should be any HTTP requirements for such a test suite,
> and I think docker & github pages should be all that is needed.
>
> We have been maintaining this test suite at DIF:
> https://github.com/decentralized-identity/JWS-Test-Suite
>
> It would be nice to clean it up a bit, which I am happy to take point on.
>
> I propose a new repo for the vc-jwt test suite: vc-jwt-test-suite, I would
> like to be a CODEOWNER, an empty repo will be fine to start.
>
> Regarding the core data model test suite, I don't believe HTTP should be a
> requirement of that test suite either.
>
> I don't plan to contribute substantially to the core data model test suite.
>
> I do plan to help with a test suite for vc-di-bbs, and I think that test
> suite should be independent of the other test suites given the unique
> challenges of JSON-LD framing and selective disclosure.
>
> It would be good to start gathering contributors for all test suites.
>
> Regards,
>
> OS
>
>
>
>
> On Sat, Apr 8, 2023 at 3:05 PM Manu Sporny <msporny@digitalbazaar.com>
> wrote:
>
>> Hi all,
>>
>> As we march towards the Candidate Recommendation phase, it's important
>> for us to have a believable test suite story. The Test Suite we used
>> in VC 1.0 and 1.1 left much to be desired, a few of those things
>> being:
>>
>> * The v1.x test suite was only run a handful of times by implementers
>> and then not used thereafter. This resulted in implementations
>> drifting in their conformance over time. If we had to do this again,
>> we'd adopt a continuous integration / testing approach where the test
>> suite would be run against all current implementations on a regular
>> basis.
>>
>> * It didn't test the "securing" mechanism for VCs, and focused on just
>> the normative data model statements. We need to test the securing
>> mechanisms this time around.
>>
>> * It required implementers to submit test reports manually, even if
>> they had an automated process for generating the test reports.
>>
>> Fast forward a few years and many of us have participated in plugfests
>> that provided better interoperability testing infrastructure that
>> overcame all of the shortcomings listed above. For example, the recent
>> Jobs for the Future plugfest demonstrated interoperability between 17
>> different issuers with an updated test suite infrastructure (see
>> slides 4 and 5):
>>
>>
>> https://docs.google.com/presentation/d/19GmJ3bLMrbVadesnkmsWaaUr-U71Y9Kr775tZvgs-xI/edit
>>
>> Thanks to Andrew Jones and Charles Lehner, we now have a VC Data Model
>> 2.0 test suite that tests 22 of the 44 normative statements in the VC
>> Data Model 2.0 specification (see example conformance report attached
>> to this email):
>>
>> https://github.com/digitalbazaar/vc-data-model-2-test-suite
>>
>> We also have test suites that can test the VC Data Integrity base
>> specification as well as each cryptosuite specification:
>>
>> https://w3c-ccg.github.io/di-ed25519-test-suite/
>>
>> While most of the implementations today integrate using the VC API,
>> the test suites are flexible enough that other "drivers" could be
>> added to support Docker-based workflows as well. These need not be the
>> only test suites that the WG employs, if others want to contribute
>> test suites, I imagine those would be welcomed discussion.
>>
>> VCWG Chairs, this is a request for some call time (main call or
>> special topic call) to consider the adoption of at least the
>> vc-data-model-2-test-suite and the di-ed25519-test-suite listed above.
>>
>> -- manu
>>
>> --
>> Manu Sporny - https://www.linkedin.com/in/manusporny/
>> Founder/CEO - Digital Bazaar, Inc.
>> News: Digital Bazaar Announces New Case Studies (2021)
>> https://www.digitalbazaar.com/
>>
>
>
> --
> *ORIE STEELE*
> Chief Technical Officer
> www.transmute.industries
>
> <https://www.transmute.industries>
>