- From: Mike Prorock <mprorock@mesur.io>
- Date: Mon, 17 Apr 2023 07:42:22 -0600
- To: Orie Steele <orie@transmute.industries>
- Cc: W3C VC Working Group <public-vc-wg@w3.org>
- Message-ID: <CAGJKSNR8t5Bwn1T1R0U-MGTeSFpMRu2Ksw_9EkLhBLJ6YesJOg@mail.gmail.com>
I think just a "reproducible build" checkbox or date of last test is fine here. We will commit to improving documentation on this test set, as well as improving the tests themselves, and would like to move forward with this approach for VC-JWT Mike Prorock CTO, Founder https://mesur.io/ On Mon, Apr 17, 2023 at 7:39 AM Orie Steele <orie@transmute.industries> wrote: > That is correct, but needs to be documented better. > > Of course the implementation could be just calling another registered > implementation behind the scenes in that case... perhaps we give > reproducible builds of the test report a "higher rating". > > Ideally we would see implementations in go, java, rust, javascript, > typescript, python and c... before we see people registering "conformance > results" from a black box implementation. > > This should be less hard, given the new JWT structures. > > We also need to consider adding negative tests... injecting "emoji", > misconfigured mandatory and optional fields, etc... > > OS > > On Mon, Apr 17, 2023 at 8:30 AM Mike Prorock <mprorock@mesur.io> wrote: > >> Cool - it looks like also, while docker is there (and would be a >> preferred way to execute for us) there is no real mandate to even use >> docker to test an implementation, as long as you checked in the appropriate >> key and output in the outputs folder you can get a test report. Am I >> reading that code right? >> >> Mike Prorock >> mesur.io >> >> On Mon, Apr 17, 2023, 07:23 Orie Steele <orie@transmute.industries> >> wrote: >> >>> Inline: >>> >>> On Mon, Apr 17, 2023 at 8:01 AM Mike Prorock <mprorock@mesur.io> wrote: >>> >>>> Orie, >>>> Thanks for this. I believe we can work with this approach, and we will >>>> contribute an implementation and help clean up and improve. It looks like >>>> a second pass of test(s) could then run on the outputs of this once VC-JWT >>>> is verified, to then verify against the core data model. Is that your >>>> thinking? >>>> >>>> >>> Yes, there are a couple different ways to process the results / grade >>> them. >>> >>> The idea is to allow implementation to produce both issuance and >>> verification results for analysis / grading. >>> >>> >>>> Also, it looks like this could be extended so that tests could run in >>>> an independent environment for those with proprietary implementations. Is >>>> that a correct read as well? >>>> >>>> >>> Yes, building on docker also gives us some tools to help consider some >>> of the tougher parts of cryptographic testing such as FIPS. >>> >>> - >>> https://ubuntu.com/blog/building-and-running-fips-containers-on-ubuntu18-04 >>> >>> There are some more advanced features we could consider: >>> >>> - publishing a conformance image that makes evaluation of the test suite >>> easier, friendlier for private environments. >>> - publishing a GitHub Action so that conformance evaluation can be >>> dropped into any CI pipeline on GitHub (even private repos). >>> >>> Some of the current pathing / data flows might need to be touched up to >>> make these easier in the future. >>> >>> I am currently mounting a data directory and assuming some pathing, >>> which feels a bit awkward. >>> >>> Mike Prorock >>>> mesur.io >>>> >>>> On Sun, Apr 16, 2023, 17:48 Orie Steele <orie@transmute.industries> >>>> wrote: >>>> >>>>> I put this together today: >>>>> >>>>> - https://transmute-industries.github.io/vc-jwt-test-suite >>>>> - https://github.com/transmute-industries/vc-jwt-test-suite >>>>> >>>>> It's a test suite built on docker, github actions and github pages. >>>>> >>>>> There are some rough parts to it, and it obviously does not run core >>>>> data model conformance tests yet. >>>>> >>>>> This test suite has no JSON-LD processing dependency. >>>>> >>>>> I believe the core data model can be tested without doing any JSON-LD >>>>> processing, but perhaps we will see if that is truly the case. >>>>> >>>>> Regards, >>>>> >>>>> OS >>>>> >>>>> -- >>>>> *ORIE STEELE* >>>>> Chief Technical Officer >>>>> www.transmute.industries >>>>> >>>>> <https://www.transmute.industries> >>>>> >>>> >>> >>> -- >>> *ORIE STEELE* >>> Chief Technical Officer >>> www.transmute.industries >>> >>> <https://www.transmute.industries> >>> >> > > -- > *ORIE STEELE* > Chief Technical Officer > www.transmute.industries > > <https://www.transmute.industries> >
Received on Monday, 17 April 2023 13:42:38 UTC