Re: Announcing canivc.com

Hi Anil,

Thank you for this feedback and for the input you gave us earlier this
year! Your input is where those lovely spider charts came from. :)

Also, I've made an issue for the filtering idea you shared here:
https://github.com/digitalbazaar/canivc/issues/25

Being able to filter the implementation list by specs implemented would go
a long way toward helping folks see the forest for the trees.

Additionally, the listing and counting across the specs is (and may remain)
a complicated endeavor, and even some basic filtering here will minimize
the "gaming" of the system, hopefully.

Thanks again for all the feedback!

Cheers,
Benjamin

On Tue, Mar 19, 2024 at 8:38 AM John, Anil <anil.john@hq.dhs.gov> wrote:

> I appreciate the work that went into this. Thank you.
>
>
>
> In particular, I appreciate that this provides a concrete and test-driven
> view into implementations and the features supported by an implementer,
> rather than self-assessment/assertion by an implementer of what they
> support.
>
>
>
> I also realize that this is a work in progress, so wanted to offer some
> (random) feedback.
>
> I like the use of spider charts to visually show which features are
> supported by a specific implementer (
> https://canivc.com/implementations/digital-bazaar/
> <https://urldefense.us/v3/__https:/canivc.com/implementations/digital-bazaar/__;!!BClRuOV5cvtbuNI!HSGVMicJqDkjuU_9oSKy8UIhqXkugv7PcHIsFlvhqplxHZgBE96dnkdkSnwCrQ35aLspXowGMCRlDnrbd1w$>
> )
>
> I like the use of the test suite reports that provide feedback on
> specifics of implementation (
> https://canivc.com/reports/did-key-test-suite/suites/did-key-create-operation
> <https://urldefense.us/v3/__https:/canivc.com/reports/did-key-test-suite/suites/did-key-create-operation__;!!BClRuOV5cvtbuNI!HSGVMicJqDkjuU_9oSKy8UIhqXkugv7PcHIsFlvhqplxHZgBE96dnkdkSnwCrQ35aLspXowGMCRl-xSr5xo$>)
> across implementers
>
> I am not sure how to interpret the number of tests associated with each
> issuer and each verifier on the home page; I am unsure of the value
> provided by the total number of tests a particular entity has chosen to
> implement.
>
>
>
> I love that you are making a distinction between failing a test and the
> test not being implemented by an entity.
>
>    - I am also wondering if the distinction should be passing/failing a
>    test (if that test is of a feature in a spec that is required) and an
>    entity making a conscious decision that there is a particular option in the
>    standard that they are supporting/not-supporting.
>    - An example of this in the VCDM 2.0 is that you are required to
>    either support an embedded proof model (Data Integrity) or an enveloped
>    proof model (JOSE/COSE). An entity making a choice to support one option
>    only for reasons that are their own is a perfectly rational one and should
>    not come with any negative implications.
>
>
>
> The above comment is also a question regarding if it is possible to make a
> distinction between tests that are required and tests that are optional
> within the test suite, and group them that way if possible.
>
>
>
> Some of the ways that I've seen people game tests is to (1) self-select a
> specific set of tests/features that they implement so that every test shows
> as passing and (2) add on a whole bunch of tests beyond what is in the
> standard/spec test suite itself to show how wonderful they are with
> everything they support in their product (more is not necessarily better)
>
>
>
> Ability to mitigate these types of gaming would be welcome with a possible
> future outcome whereby I have the ability to select the particular test
> suites that are of interest to me, and have the ability run them against a
> (set of) implementers.
>
>
>
> Going forward, I would love to be able to use these tests to find answers
> to questions like the following:
>
>    1. Show me the subset of implementers that can issue VCDM 2.0 + Data
>    Integrity credentials + proof sets
>    2. Show me the sub-set of implementers that can issue VCDM 2.0 +
>    JOSE/COSE credentials
>
> Show me the intersection between (1) and (2)
>
> Show me the entities that if I choose to go down (1) or (2), have products
> that can verify those credential/proof combos.
>
>
>
> Needless to say, this is a great and useful and I am looking forward to
> the incorporation of additional test suites to provide a more comprehensive
> picture of decentralized identity implementations.
>
>
>
> Best Regards,
>
>
>
> Anil
>
>
>
> Anil John
>
> Technical Director, Silicon Valley Innovation Program
>
> Science and Technology Directorate
>
> US Department of Homeland Security
>
> Washington, DC, USA
>
>
>
> Schedule a meeting with me (30 minutes; non-DHS people only)
> <https://outlook.office.com/bookwithme/user/6250c4b6cae94d549b6db87b72b0b6d5@hq.dhs.gov?anonymous&ep=plink>
>
> Time Zone: UTC-05:00 (US Eastern Time)
>
>
>
> Email Response Time – 24 Hours or more; I sometimes send emails outside of
> business days/times because it works for me; please do not feel any
> obligation to reply to them outside of your normal working patterns.
>
>
>
> [image: A picture containing graphical user interface Description
> automatically generated] <https://www.dhs.gov/science-and-technology>[image:
> /Users/holly.johnson/Library/Containers/com.microsoft.Outlook/Data/Library/Caches/Signatures/signature_1972159395]
>
>
>
> This document contains pre-decisional and/or deliberative process
> information exempt from mandatory disclosure under the Freedom of
> Information Act, 5 U.S.C. 552(b)(5). Do not release without prior approval
> of the Department of Homeland Security.
>
>
>
> *From:* Benjamin Young <byoung@digitalbazaar.com>
> *Sent:* Monday, March 18, 2024 5:12 PM
> *To:* W3C Credentials Community Group <public-credentials@w3.org>
> *Subject:* Announcing canivc.com
>
>
>
> *CAUTION: *This email originated from outside of DHS. DO NOT click links
> or open attachments unless you recognize and/or trust the sender. Contact
> your component SOC with questions or concerns.
>
>
>
> Greetings, Verifiable Credentials WG and Credentials CG,
>
> (Sorry for the repost! I BBC'd this list which I realized too late would
> make responding from here harder... My apologies! Replying to this email
> should avoid bounces for anyone not in the VC WG.)
>
>
>
> Today, I'm excited to announce the alpha release of the Verifiable
> Credentials community compliance dashboard: https://canivc.com/
> <https://urldefense.us/v3/__https:/canivc.com/__;!!BClRuOV5cvtbuNI!HXhyD5gT7iMgKthhckB6bsvjiexeovsBcDwYDv4l_qqttGjj6PusPnPM60hKohnjbgMs4mbJWc1TuzwU1tO4dwXt$>
>
> We interviewed over 40+ people earlier this year to help craft a site that
> presents the state of Verifiable Credential compliance on a single site.
> Our hope is to provide a place where anyone can review the progress of the
> various implementers who are working to reach compliance with one or more
> VC related test suites.
>
> The current focus of the site is on the in-progress W3C VC Working Group
> and Credential Community Group specifications. Creating a single community
> dashboard from the aggregated results of the test suites provides a much
> needed window into the present compliance and the interrelatedness of the
> specifications.
>
> The site provides a list of the specifications being tested, their test
> reports, the results of each test suite, the list of implementers and
> implementations being tested, as well as spider charts per-implementation
> showing the focus of that specific implementation.
>
> As always, your feedback is very valuable to us. We hope that this site
> helps present the state of the ecosystem, encourages others to join the
> compliance activities, and helps developers and decision makers better
> understand what code and services are out there for building and deploying
> Verifiable Credentials as part of their products and services.
>
> Thank you for all you're doing to contribute to this ecosystem!
>
> Benjamin
>
> --
>
> https://linkedin.com/in/benjaminyoung
> <https://urldefense.us/v3/__https:/linkedin.com/in/benjaminyoung__;!!BClRuOV5cvtbuNI!HXhyD5gT7iMgKthhckB6bsvjiexeovsBcDwYDv4l_qqttGjj6PusPnPM60hKohnjbgMs4mbJWc1TuzwU1iKWVUUj$>
>
> Developer Engagement Engineer
>
> Digital Bazaar
> <https://urldefense.us/v3/__https:/digitalbazaar.com/__;!!BClRuOV5cvtbuNI!HXhyD5gT7iMgKthhckB6bsvjiexeovsBcDwYDv4l_qqttGjj6PusPnPM60hKohnjbgMs4mbJWc1TuzwU1kbn5FlP$>
>


-- 
https://linkedin.com/in/benjaminyoung
Developer Engagement Engineer
Digital Bazaar <https://digitalbazaar.com/>