Re: VCWG Test Suites

I would like to contribute to the vc-di-bbs in anyway I can help.

Brian

On Fri, Apr 14, 2023 at 6:37 PM Mike Prorock <mprorock@mesur.io> wrote:

> Orie,
> Thank you for volunteering to clean that test suite up - it is very clean
> and we can work with that.
>
> I too would prefer a simple docker approach where implementers provide an
> app in docker that takes a test input and provides an output and that does
> not require any network or other connections.
>
> I would prefer to have a separate test set for VC-JWT, and we will
> contribute an independent implementation for all items covered under that
> scenario.
>
>
> Mike Prorock
> CTO, Founder
> https://mesur.io/
>
>
>
> On Thu, Apr 13, 2023 at 8:32 AM Orie Steele <orie@transmute.industries>
> wrote:
>
>> Manu, thanks for taking the time to write this up.
>>
>> I'm interested in working on a test suite for securing W3C Verifiable
>> Credentials with JWTs.
>>
>> I don't think there should be any HTTP requirements for such a test
>> suite, and I think docker & github pages should be all that is needed.
>>
>> We have been maintaining this test suite at DIF:
>> https://github.com/decentralized-identity/JWS-Test-Suite
>>
>> It would be nice to clean it up a bit, which I am happy to take point on.
>>
>> I propose a new repo for the vc-jwt test suite: vc-jwt-test-suite, I
>> would like to be a CODEOWNER, an empty repo will be fine to start.
>>
>> Regarding the core data model test suite, I don't believe HTTP should be
>> a requirement of that test suite either.
>>
>> I don't plan to contribute substantially to the core data model test
>> suite.
>>
>> I do plan to help with a test suite for vc-di-bbs, and I think that test
>> suite should be independent of the other test suites given the unique
>> challenges of JSON-LD framing and selective disclosure.
>>
>> It would be good to start gathering contributors for all test suites.
>>
>> Regards,
>>
>> OS
>>
>>
>>
>>
>> On Sat, Apr 8, 2023 at 3:05 PM Manu Sporny <msporny@digitalbazaar.com>
>> wrote:
>>
>>> Hi all,
>>>
>>> As we march towards the Candidate Recommendation phase, it's important
>>> for us to have a believable test suite story. The Test Suite we used
>>> in VC 1.0 and 1.1 left much to be desired, a few of those things
>>> being:
>>>
>>> * The v1.x test suite was only run a handful of times by implementers
>>> and then not used thereafter. This resulted in implementations
>>> drifting in their conformance over time. If we had to do this again,
>>> we'd adopt a continuous integration / testing approach where the test
>>> suite would be run against all current implementations on a regular
>>> basis.
>>>
>>> * It didn't test the "securing" mechanism for VCs, and focused on just
>>> the normative data model statements. We need to test the securing
>>> mechanisms this time around.
>>>
>>> * It required implementers to submit test reports manually, even if
>>> they had an automated process for generating the test reports.
>>>
>>> Fast forward a few years and many of us have participated in plugfests
>>> that provided better interoperability testing infrastructure that
>>> overcame all of the shortcomings listed above. For example, the recent
>>> Jobs for the Future plugfest demonstrated interoperability between 17
>>> different issuers with an updated test suite infrastructure (see
>>> slides 4 and 5):
>>>
>>>
>>> https://docs.google.com/presentation/d/19GmJ3bLMrbVadesnkmsWaaUr-U71Y9Kr775tZvgs-xI/edit
>>>
>>> Thanks to Andrew Jones and Charles Lehner, we now have a VC Data Model
>>> 2.0 test suite that tests 22 of the 44 normative statements in the VC
>>> Data Model 2.0 specification (see example conformance report attached
>>> to this email):
>>>
>>> https://github.com/digitalbazaar/vc-data-model-2-test-suite
>>>
>>> We also have test suites that can test the VC Data Integrity base
>>> specification as well as each cryptosuite specification:
>>>
>>> https://w3c-ccg.github.io/di-ed25519-test-suite/
>>>
>>> While most of the implementations today integrate using the VC API,
>>> the test suites are flexible enough that other "drivers" could be
>>> added to support Docker-based workflows as well. These need not be the
>>> only test suites that the WG employs, if others want to contribute
>>> test suites, I imagine those would be welcomed discussion.
>>>
>>> VCWG Chairs, this is a request for some call time (main call or
>>> special topic call) to consider the adoption of at least the
>>> vc-data-model-2-test-suite and the di-ed25519-test-suite listed above.
>>>
>>> -- manu
>>>
>>> --
>>> Manu Sporny - https://www.linkedin.com/in/manusporny/
>>> Founder/CEO - Digital Bazaar, Inc.
>>> News: Digital Bazaar Announces New Case Studies (2021)
>>> https://www.digitalbazaar.com/
>>>
>>
>>
>> --
>> *ORIE STEELE*
>> Chief Technical Officer
>> www.transmute.industries
>>
>> <https://www.transmute.industries>
>>
>