[MINUTES] DID WG Special Topic 2025-08-20

DID Working Group Special Topic Call - 2025/08/20 Summary

*Topics Covered:*

   -

   *DID Resolution Mocha Test Suite Review:* The group reviewed a newly
   developed Mocha test suite for DID resolution, built upon existing VC and
   CCG test suite frameworks. The initial implementation focuses on GET
   requests against sample DID resolvers.
   -

   *Test Suite Design Discussion:* A significant portion of the meeting
   debated the optimal design of the test suite. The core discussion centered
   around whether the suite should exhaustively test all DID methods or focus
   on a smaller set of core functionalities.
   -

   *Resolver Registration and Test Case Selection:* The group discussed how
   DID resolver implementers should register their resolvers and select the
   DID examples used in testing. There was strong agreement that resolvers
   should *not* mark their own homework, and that the test suite should
   provide a mechanism for the group to validate resolution results. The
   approach of using a limited set of example DIDs for core functionality
   testing was favoured over an exhaustive, per-DID-method approach.
   -

   *Test Case Structure and Organization:* The group explored alternative
   data models for organizing test cases, considering grouping by DID method
   versus grouping by general test categories (e.g., valid resolution,
   unsupported method). The group leaned towards categorizing by generic test
   categories to avoid inadvertently testing DID method specifics.
   -

   *Handling of Resolution Options:* The need to accommodate DID methods
   requiring specific resolution options (e.g., versioning in did:btc) was
   acknowledged. The consensus was that these options should be provided by
   the test cases themselves.

*Key Points:*

   - The initial DID resolution test suite provides a foundation but
   requires further development of test cases.
   - The group strongly prefers a test suite focused on core DID resolution
   functionality, rather than comprehensive DID method testing.
   - A key goal is to avoid placing an undue burden on resolver
   implementers by requiring support for all DID methods.
   - The test suite should use a set of standardized test DIDs to ensure
   consistency and fair evaluation.
   - The method for selecting and providing test DIDs and resolution
   options requires further refinement.
   - The group agreed on a process where resolvers register their endpoints
   and provide the DIDs they can support within the test suite.
   - The group will evaluate the results from this process and explore
   different data models for reporting resolution results.

Text:
https://meet.w3c-ccg.org/archives/w3c-ccg-did-wg-special-topic-2025-08-20.md

Video:
https://meet.w3c-ccg.org/archives/w3c-ccg-did-wg-special-topic-2025-08-20.mp4
*DID Working Group Special Topic Call - 2025/08/20 09:55 EDT - Transcript*
*Attendees*

Benjamin Young, Joe Andrieu, Kevin Dean, Manu Sporny, Ted Thibodeau Jr,
Will Abramson
*Transcript*

Will Abramson: Not too bad.

Kevin Dean: I was getting worried there for a minute.

Will Abramson: How is everybody? I guess there's only five. I was worried
too. I got auto to set up the IRC, but actually we're not going to do I
forgot that. We're just all in here. Is that correct?

Manu Sporny: Yes. …

Will Abramson: We don't need to scribe.

Manu Sporny: It's recording in transcribing. so I think we're good to go.

Will Abramson: Are we expecting Benj at the top?

Manu Sporny: Yes. Let me go ahead and ping him. Let me All right,…

Kevin Dean: Yeah, just confirm that it's recording and…

Kevin Dean: transcribing because nor no. I see the indication.

Will Abramson: Yeah. Yeah,…

Kevin Dean: Never mind. I see the indicator. Normally, I get something a
little more prominent than that, but I do see it, so never mind.

Manu Sporny: I just pinged him. Hopefully, he'll be here shortly.

Will Abramson: Marcus, I think he's on holiday with his family this week.

Will Abramson: I mean, do you know if Benjamin has something to show us?
I'm expecting Cool.

Manu Sporny: Yeah, he does. Yeah. I mean, we've got something set up and I
was able to raise an issue.

Will Abramson: Cool. …

Manu Sporny: I can probably at a high level kind of talk through,…

Will Abramson: let's give him another minute. I think it's fine.

Manu Sporny: but Yeah.

Will Abramson: Maybe it is worth just starting with. I don't know.

Manu Sporny: Yeah. let me go ahead and share my screen. So, what we have at
this point, is a did resolution mocha test suite.

Will Abramson: Could you drop in?

Manu Sporny: So the frameworks set up. Benjamin's been working on this over
the past two weeks. it uses the same kind of, basic setup, that we have for
the other test suites.

Manu Sporny: There's links won't be yep,…

Will Abramson: That's fine. I just want to open it.

Manu Sporny: Absolutely. here we go. there's a link and…

Will Abramson: Great. Mhm.

Manu Sporny: it's got this, standard stuff, table of contents, installation
instructions, how you run the test suite, and then how you get your
implementations integrated into the test suite. let me go ahead and show
what the output let's see this one. So the output basically it currently
looks like this. so it's a little messy, right?

Manu Sporny: But you can see that we've already integrated the Danube tech
stuff and…

Will Abramson: parents.

Manu Sporny: you can see that it's already running against the Danny Tech
implementation. So we can talk about some of the details. Is there a bunch
of did methods that have been kind of put in the current tests show adanium
tech either properly resolving it not. Benjamin, I just started kind of
going over the basic test suite,…
00:05:00

Manu Sporny: but perhaps you can take us through this.

Benjamin Young: Yeah, sure.

Benjamin Young: Sorry to be late. this is just a pick up from the other
similar VC and CCG test suites. It's based on the same Mocha framework.
There is one very naive test right now that just does get requests against
all the sample did resolver.

Will Abramson: Nothing.

Benjamin Young: So it's literally just the same ones that did resolver has
in their sample select list. and as you can see the table layout is jacked
up which is something we need to talk about because ds are massive. and
they're not grouped at this point either. and I don't know if we want that
necessarily, but what we would end up having is as we add additional
resolvers, we'd have more columns to the right.

Benjamin Young: And we'll do something about the extremely wide left hand
column. So, right now, this is just Danube. Yeah, that's so Manu. What you
scroll down to is at risk section. that's going to be there until I can
shut it off but that basically shows that there's only one implementation
passing. So once we have two passing that'll go ahead.

Will Abramson: So I have a question…

Will Abramson: which is so here we have just dids and then whether it
passes or not and passes it resolves and returns like a valid resolution
result with a conform and did document. Is that correct?

Benjamin Young: Yeah, Monu, can you go back to the code? the test itself is
five lines. it's what this was was getting it all set up and wired into
GitHub actions and…

Will Abramson: Mhm. Yeah.

Benjamin Young: the reporter and everything else. it's Yeah.

Will Abramson: Go for it.

Benjamin Young: Yeah. Go ahead.

Manu Sporny: I raised an issue. I think I know what you're getting at,
Will. I raised an issue that we should talk about on the data resolution
test suite.

Will Abramson: I guess my question is I think this is one test case, and
probably I don't know what we think but I was thinking maybe a did resolver
only needs to supply one passing test and maybe it's one pass for every did
method.

Manu Sporny: Yeah, that's the issue.

Will Abramson: Yeah. Right.

Manu Sporny: Yeah, that's the issue I raised.

Will Abramson: Okay. Yes, sure.

Manu Sporny: I'd like to talk about it today, but go ahead, Think you were
just mostly talking about infrastructure, not the specific tests.

Benjamin Young: right? Yeah.

Benjamin Young: So, the task I'd taken on was getting this set up and
functioning in terms of publication. we had tried to make it clear that
Digital Bazaar wasn't going to carry the water on this one. So we wanted to
set it up and have somebody else tease out the tests. I did find there is a
older Cypressbased resolution test suite that has a whole bunch of example
dates. I didn't try and run it because Cyprus is open core and I didn't
know how much of that was going to end up being something we'd have to pay
for.

Benjamin Young: But there are tests in this repo that could be consulted
and moved over potentially if they're even close to what's wanted now. but
it's really going to be on the group to figure out…

Will Abramson: Mhm.

Benjamin Young: what is a fully passing D resolution experience per method.
And the other thing that another either separate test or mixed in together
would be whether or not the FA failure cases are handled properly as well.

Joe Andrieu: What's it

Benjamin Young: Because I said, it's a super minimal test suite, so it's
not doing anything really. but some of these could be just failed get
requests where the resolver choked and died with a 500 error or it could be
that the resolver is coming back with a meaningful error response from the
did method and that has content written up about it in did resolution that
could be properly tested and whatever.

Benjamin Young: So really it's just about hunting down the musts and making
sure the tests map to that across however many did methods we're going to
test. Go ahead

Manu Sporny: Yeah, I think it's important to talk about this stuff, but I
think we're going very deep very quick instead of just looking at the
infrastructure and what's there. cuz Benjamin we haven't even talked about
how are these things kicked off GitHub actions how are the reports
generated when are they generated I think maybe let's start a bit more high
level before we jump into the very specifics

Will Abramson: No. Yeah,…
00:10:00

Will Abramson: I guess I say I agree with that. I would like to see at some
point in this call the process by…

Will Abramson: which someone can add a resolver to this in its current
state. Is that possible?

Benjamin Young: Yeah, sure.

Benjamin Young: Yeah, and the questions were added into why doesn't it test
X, Y, and Z? So, that's why we went down this road. if you go back to that
repo this pulls from if you go to the package JSON file see where this is
connected go down to dependencies. So there is the VC test suite
implementations it's pulling from the CCG's VC test suite implementations
thing for now that can be changed to the main W3C one. And then if you go
there, Yep. and then go to Danube,…

Benjamin Young: you will see that they have that line. Yep. And that's all
the more that's needed is an endpoint URL tagged with just did resolution
for the moment…

Will Abramson: Right.

Benjamin Young: because where Marcus and I had gotten over email was that
just having one thing stated this endpoint did resolver and then letting
the test suite just hammer it with all available from the test suite side
dids and that responds was…

Will Abramson: That's good…

Benjamin Young: what we had discussed but that's all it's doing now.

Will Abramson: but I was expecting in here did that this resolver is going
to resolve this did right. I don't know I'm missing that but it's kind of

Benjamin Young: Go ahead Mike

Manu Sporny: Yeah, let's just go into the issue because we keep coming back
to it.

Will Abramson: Mhm.

Manu Sporny: I'm going to go to the issue I raised. So, there's a mismatch
in expectations, I think that's what's leading us to keep coming back to
this question. when we had discussed it last time, I think we had expected
to see something like this. So I've issu raised an issue on the Mocha
resolution test suite that specifies for each did resolver when you
register resolver are going to say these are the DIDs that I want you to
use for these specific tests. it's not meant to be a DID method test suite.
That's I think what it comes across as right now.

Manu Sporny: What we want is a did test suite that actually tests the
resolution stuff, but again, as Benjamin said, he did not sign up to do
that work. we have to find who's going to take this over and then figure
out how a resolver, suggests what dids can be used with that resolver and
for which test cases. let me stop there just to say,…

Will Abramson: So, engine

Manu Sporny: will I know you're looking at it and…

Manu Sporny: you're like, this is That is why it's not making sense to you.
I think we need to make a modification to it. which goes beyond what I
think Benjamin, had signed up to do. Anyway, I'll be quiet

Benjamin Young: You're fine. Yeah, I think actionably where the plan had
been was that the infrastructure would be set up and then that there would
be a group determined that would work on the test suite in relationship to
the data working group to determine…

Will Abramson: Yes.

Benjamin Young: how best to come at either this approach or the approach
that rcus and I determined. The reason Marcus and I didn't go the route you
describe here, Manu, is it's like having the student say, "This is what
Test me on what I know." And then it's like, "Hey, look, you passed what"
And the intent was to instead say, "Here's a bunch of didds. We don't
actually expect that you'll pass them all. but you should resolve all the
ones that you can resolve." And we're only testing the resolution response.
so relatedly the resolvers will not ever all pass all the tests or rather
not respond positively to a test for every did method because they may not
support them all.

Benjamin Young: We could this ultimately though is the decision of both the
working group and whoever is going to write this and…
00:15:00

Benjamin Young: really there needs to be better representation from
resolution implementers which digital bazaar notably is

Will Abramson: Cool. Thanks for Yeah,…

Will Abramson: I have a couple of things. so yeah, totally appreciate that
you guys didn't sign on to do this work and you just ported over what we
had, right, and showed it integrated in together. And I will say that, I'm
pretty busy, but I'm happy to take on some of this work. I think is
interesting and worth work. I'm happy to explore it and I'll also volunteer
Beno even though he's not on the call. I've asked him if I can. He hasn't
replied yet, but I talked to him about this before, He's an invite expert
with some decent technical chops. I think he has worked on test suites like
this before. So hopefully me and Beno and anyone else maybe we can pull you
into Benjamin.

Will Abramson: and then the second thing I wanted to say is I appreciate
what you're saying about marking your own work but I do not think it is
realistic for the did resolver test suite to have exhaustive cases for all
the did methods out there but maybe some middle ground is I don't know how
we specify this and obviously I don't think it can be normative but really
did method specifications that are defining test cases

Will Abramson: that are for this is a valid not found did this is a method
not supported or maybe not methods but not supported but then did resolvers
should say look this resolver is designed to resolve did method x and you
need to get the test cases from did method x and run them through this did
resolver test suite I mean obviously the did method as defining those test
cases might define in more detail the results that you're expecting And I
think we've discussed this before. I don't think the D resolution cares
about the deep details of those results, but it does care about okay, this
did method is saying this is valid and it should have a resolution result
with a valid conformant did document in it. I feel like that is the right
place to define those things if we aren't letting did resolvers define them
because I get it, right?

Will Abramson: that is like marking your own homework at some so I'll stop
that.

Manu Sporny: Yeah, a big plus one to what you're saying, I'm wondering if
Benjamin was not on the call where we had this discussion. it sounds we
definitely talked about this being not the way we wanted to do things like
that. we definitely said this is we do not want to do this. And so I think
maybe Benjamin you weren't on that call. and that's why it kind of got set
up in that and I went into in the issue here when we have found did work
when we take this approach we will get a lot of complaints from
implementers because they will start failing tests that they never intended
to pass and they don't want to show up as having red X's behind beside
their implementations so Will I feel very strongly that

Manu Sporny: this is definitely not the way we want to do this. this is the
wrong way to go about the test suite. I think it's going to create a whole
bunch of complaints from did resolution implementers and cause some people
to just not add their implementations to the implementation. so, a plus one
to that. what you're saying, I think we need to move away from marking your
own homework thing. The way that we had discussed addressing that the last
time was that we are just telling them to put the DIDs that they support or
specific calls in …

Will Abramson: Mhm.

Manu Sporny: but we are in full control of saying whether or not the
response was valid. Meaning we will have a JSON schema that we run against
the DID document.

Manu Sporny: we will have a JSON schema that we run against the did
resolution result and they don't get to mark their own homework when it
comes to that. it's like and that is the thing we're supposed to be testing
here, we're supposed to be testing the DID resolution API request and
responses. We're definitely not supposed to be testing DID methods here, so
they do get to come to us and say the DID methods I support. But that's
kind of the same thing as coming to us and saying these are the VC API
endpoints I don't support endpoint I'm not set up to do that. I might have
a philosophical disagreement with that particular, thing. And so I'm
choosing not to do that.

Manu Sporny: So all we can really do in that case is basically say whatever
you have…
00:20:00

Manu Sporny: if you implement a did resolution at all for any did method we
want you as a part of the test suite and we promise we're not going to make
you look bad …

Manu Sporny: if you choose not to implement did method I'll stop there.

Will Abramson: Great. Thanks,…

Will Abramson: Benjamin.

Benjamin Young: Yeah, this list of supported bits that are valid that's
going to be a massive list for every single resolver with at least one per
method as far as I understand it.

Benjamin Young: Versus hey, we support these methods and then there's real
tension here because obviously the did methods underneath on the other side
beyond the resolver is the thing that's going to actually and have to keep
it available and resolvable. So yeah I don't know but ultimately what's
highlighted here is the test suite because running some JSON schema against
a response is trivial. We can do that but the test suite itself is
essentially going to have to iterate over all these values and then check
the expected responses.

Benjamin Young: which is possible. but that payload is potentially getting
very unwieldy very quickly.

Benjamin Young: Just some thoughts.

Will Abramson: Great. Thanks.

Will Abramson: Kevin

Kevin Dean: Yes, I would expect some grumbling in the initial deployment of
any testing framework simply because there already did methods out there
that are guaranteed we can pretty much guarantee some of them will fail.
but as we go through time it would be very surprising to me and that to see
a dead method come in for testing that fail the tests because if the tests
are made publicly available even if we stand up a server that can run the
tests on your behalf as if I were a DID method author I would not be
submitting something without having run

Kevin Dean: the test myself and identified all of the errors and areas of
discrepancy and fixed it so that on submission it actually runs because
there is an interest on the part of the DIG method implementers to have
something that is compatible and if the test suite is what determines
compatibility and det and determines adoption they've got the incentive to
ensure at multiple points during the development of their DID method that
it can pass the tests. I don't see a long-term problem with it. I certainly
see a short-term one for existing methods,…

Kevin Dean: but not a long-term one.

Will Abramson: Great. Thanks,…

Will Abramson: Man.

Manu Sporny: Yeah, this is why we're here, but we're definitely still
miscommunicating. Benjamin, you said that I don't think it's going to get
long at all. So, I'm wondering where that, is coming from. Meaning that
again, we're DID resolution. We're not testing DID methods. I think if we
say you have to did web or you have to support did fu and it is going to
show up for let's do a concrete example digital bazaar I think Benjamin we
do intend to submit a resolution a resolver meaning for did key or
something like that because we don't have enough implementations right now
and we really want to help this group get to

Manu Sporny: some kind of multiple implementations level we're hoping that
other people do more thorough implementations of bid resolution but I think
digitalizar is going to submit a resolver at some point in the next year we
will not submit a resolver if we are going to be showing up as failing 90%
of the tests like it's just because we want this to go up on the can IBC
website, we want it as another resolution thing. And if it shows up that
we're failing 90% of the DID resolver test suites because there are a whole
bunch of DID methods that we don't want to support and we're never going to
support we're just not going to submit our implementation, So, I again like
if we think that we are going down this route, we are very unsupportive of
it. I just want to be very clear about that.
00:25:00

Manu Sporny: is this list going to get out of hand? I don't understand how
it could. So, for example, if digital bazaar were submitting something, we
would just did web and did key as the things and that should get full
coverage, I would imagine, for the did resolution test suite. we might not
get some of the more advanced, examples, but I would imagine we might add
another four lines for the variations that we need. but certainly not we
get a very long list here. I think we would just have variations of did web
and We would probably just did web for the vast majority of these and call
it a day.

Manu Sporny: so I'd like to understand, I guess, Benjamin, why you think
this is going to get out of hand. and Kevin, I didn't quite understand what
you were saying about people are going to make sure that they align for
example if we said the thing I don't understand Kevin is I think you're
saying we'll just settle on did key or did web and everyone will get their
resolvers to align with that.

Manu Sporny: Some people might have serious issues with did web and we
can't really push that, on did resolver implementers. some folks, for
example, might just be implementing a Bitcoin method or an Ethereum based
method and that's all they want to support. And if in the test suite we
take a position on you have to support did web or…

Will Abramson: Mhm. Yeah.

Manu Sporny: you have to did key, I also see that as kind of like an
untenable position. let me stop there.

Kevin Dean: No, just to cl No,…

Kevin Dean: just to clarify that they will ensure that their test that they
should ensure that…

Manu Sporny: Yeah. for the dead methods they choose.

Kevin Dean: what they submit will pass the tests for the features that they
wish to implement. so if they're not so for their methods, if they're…

Manu Sporny: Is that okay?

Kevin Dean: if they're not supporting DID web,…

Manu Sporny: Plus one.

Kevin Dean: they're not supporting DID web, but what comes out of the
resolver should pass for what they are supporting should pass the tests
that are testing what they claim to support. That's all. I

Manu Sporny: We're aligned.

Will Abramson: So I have a comment which I kind of echo a lot of what Manny
was saying to you Benjamin for me we are testing did resolution spec really
people need i to provide one test case one did that is valid when using
this resolver can resolve to a valid resolution result with a conformant
did document just one right because that proves that they can do
resolution. what we might want to do is maybe split up this data model a
little bit more so that people provide one valid for every did method that
their resolver supports. So you might have actually supported methods with
a map of the method name and then all the test cases for that method right
BTC here's a valid did web did.

Will Abramson: I mean most for me I don't know about other people I think
most resolvers are going to support one or a handful obviously the
universal resolver is a edge case but a lot of people just caring about
resolving the divids that they want they want a resolver that they can use
in their systems right that's how I see it anyway so maybe it's worth
breaking up the data model…

Will Abramson: because then you could say this resolver supports these five
did methods and has, full test coverage across all of them. that would be
pretty cool to know, but I don't know if it's worth it. So, this is what
people think. Benjamin. Mhm. Yep.

Benjamin Young: So to kind of clarify some of the language that we're sort
of circling around,…

Benjamin Young: we're really just talking about where the test fixtures
come from, not is the test code itself should be the same where you get a
URL and you test the response JSON.

Will Abramson: Mhm.

Benjamin Young: So the fixtures could be defined like they are here. but
the reason I said this would get really long, is because I was basing my
knowledge off the universal resolver and if you go back, monu, to the test
suite results. that list is the drop-down list from the values of the
drop-own list at the universal resolver. And my understanding was that
Danube Tech at some point in their history believed all of those would
result because they wouldn't have put them up there as samples. So my
expectation was that 80 to 90% of these would resolve. The ones that didn't
meant that those systems were down or no longer in existence or whatever.
00:30:00

Benjamin Young: and that's possibly the case but a quick skim looks like
it's more like 50% of these fail. so the point about fixtures being these
example wherever they come from are did method side of the house not from
the resolvers. So I wonder if it begins to point towards something like
example.com or any of the Ayanna example domains where we at least
encourage if not require D methods to have an example did that they either
state or they're consistently the same shape in so far as that's possible.

Benjamin Young: that then the did resolver ecosystem can always resolve or
always fail on or whatever and then if that fails then the method itself is
probably no longer a thing. that's one idea. I don't really care where the
fixtures come from but I do hear you Manu about it. it's going to look bad
like it does for the universal resolver right now. and again, this was
something that Marcus had proposed we do to simplify the suite. So, I'm sad
he's not here to explain. so wherever those fixtures come from, they are
being provided ultimately by the method. The resolver was given the
expectation that they would resolve. In this case, these were all supposed
to be successful cases.

Benjamin Young: and the sand underneath that house is going to move just
given the passage of time even if those fixtures are defined statically in
the test suite or more manageably by ver idiosyncratically per resolver.
Even if that's the case, the resolver is still at the behest of the method
provider to keep the thing up and make sure it's still valid and whatever.
So, those are the kind of three moving parts. Writing a test suite that
leans on fixtures from wherever is not a problem. and even refactoring this
to get them out is not a problem.

Benjamin Young: And grouping them by method was a feature that was already
becoming pretty apparent in here.

Will Abramson: thanks man.

Benjamin Young: So that's it.

Manu Sporny: Plus one. It sounds like we're converging. right. So agree
with Benjamin that trying I think this is one of the reasons we kind of
depend on the resolver to tell us which did methods they want to use. for
example we may end up providing a DID key that you can use any other one
but you can also use this if you're doing it with test did web is the
better example there where we might want to provide a did web that we know
that we're in control of that's always resolvable that we can decide what
the content is in there that they can choose to use it or they can choose
to use something else.

Manu Sporny: If they choose to use something else, then it's between them
and the did method to make sure that that thing constantly resolves because
we will be testing on a weekly basis or whatever some cadence and if they
fail to resolve it then it's going to show up at failing the resolution
test. okay I think Yeah. Yeah. And so I think Benjamin the language around
test fixtures these are just inputs into a test that we control meaning and
will I think the stuff that you and Beno are going to be writing you'll
need to figure out how many different variations of a did method you're
going to need for all the different did resolution tests.

Manu Sporny: So there, I don't know, 70 80 of them or something like that.
you're probably going to use the same input did web document for let's say
20 tests or 30 tests, right? Because for a particular test, you're just
wanting the method to resolve and then maybe you're switching up the
options, you're passing the resolver or changing the resolution result or
something like that. as an option that's passed to DID resolver to give you
something slightly different. But the core document the core did method
that's resolved kind of stays the same. It says input into the test. It's a
parameter that you are passing like these are all effectively parameters
that you're going to be passing for different tests.
00:35:00

Manu Sporny: and this kind of tells you if I'm doing a test where I need
the result to be that, this is an unsupported method. because there is a
normative statement in the dead resolution spec around getting a result
back. That's, a non-supported method and you must return, this value. you
have to then give that input to the resolver so you get back something that
you can test against. So this is clearly not set in stone. I think it's
going to be up to you and Vango to figure out how to break things up. you
had suggested doing it per did method I can see the benefit of doing that.

Manu Sporny: I'm also a bit concerned about things like Marcus' resolver. I
don't think we should optimize for Marcus' resolver. it's great and it does
everything, but let's not presume that every other resolver is going to do
that. you said I don't think we should be optimizing for a resolver
resolving, 50 different dip methods. We should optimize for one to a
handful. and I think that's how we get a lot more people willing to
participate in the test suite. the other thing I want to make sure that we
don't do is we don't fall into the trap of making this a it is not a DID
method test suite. That is a separate thing. There's going to be a DID
method working group. They're going to be doing, test suites on whatever
DID methods they end up, standardizing.

Manu Sporny: if that work starts and so that is not Our job here is to test
the interfaces for data resolution and…

Manu Sporny: I think and that's it.

Will Abramson: Cool. I think that mostly makes sense.

Will Abramson: Some of the things that I'm hearing that I don't understand
is you're saying that for example that valid list we might take those
valids and run them through a series of tests, And get results. I was
imagining much more that each of those flags on the left right supported
did valid that is a flag for a test and we are putting in a did that is
going to go through that test and should meet the test case right it's
almost like I don't know if 80 is too many but we might have a bunch of
those and ideally each of those flags would be for a conformance statement
from the spec but maybe

Will Abramson: I'm thinking about that wrong but that's kind of how I was
imagining and then that's why it wouldn't be much more valid for this this
is valid for this did method this did is not found so I don't know whether
I'm just thinking about that wrong I'd love to hear what people think the
other thing I wanted to say is there are some cases like in did BTC1 for
example we don't have a did there is no such thing as not found in our
world right So that's just a flag. not so I don't know how he handle those
things. I did methods don't support certain types of error codes but I
think it's fine. We probably just don't put that test case in. The other
thing I wanted to flag did BTC1 and maybe we need to consider this did BTC1
really requires often we use resolution options to provide material that is
critical for resolving the DID.

Will Abramson: and I don't know how we navigate that but there are cases
where some did resolution tests require specific kinds of resolution
options do you handle version correctly for example. but I guess maybe
those resolution options would be provided by the test case itself. it
would be like okay this is that can be resolved at version two for example.
But then also the problem with that is the for example did methods use
different version ids right so I think that also has to be something really
that's provided by the resolver saying okay this did with these resolution
options and the test would just be to check that resolution options has a
version ID and then run it through and check that the resulting did
document or really I think did document metadata has the same version ID
your test that you would
00:40:00

Will Abramson: So I know that complicates things quite a lot but anyway
Yeah. Yeah.

Manu Sporny: No, I don't think it complicates. I mean, we need to do that,
Meaning meaning we need to be able to pass options where it makes sense to
do so. And I mean, I think that's largely up to you, Will, and Bango to try
to figure it out. you're going to have to go through and do an analysis of
the, tests and then see the variations you need. going back to your first
question though, if we have 80 tests and we have 80 things on the left
here, I don't think we will end up there. I hope we don't end up there. if
we end up there, I think we're doing it wrong.

Manu Sporny: theres certain there I think and we haven't done the analysis
yet but once we do the analysis I think there will be a number of buckets
where you need to use one did or another right and then there may be
subbuckets within there where you actually do need to pass parameters and
so my hope here is

Manu Sporny: that, maybe we get to 10 to 15 big buckets and then in those
we,…

Joe Andrieu: Yes.

Manu Sporny: if they're more specific ones for the version ID thing, maybe
we do have, an entry for version ID. So maybe each parameter gets, a bucket
here and then we say although, and we say this is the value. so we have a
version ID test, And so we're for someone that's registering their DID
resolver. They come in and they go for the version ID tests, Here's the
version you should use for a successful,…

Will Abramson: Mhm.

Manu Sporny: version, response. and then maybe any other options that the
did method resolver, needs.

Manu Sporny: But I think that at that point we're down to kind of
fine-tuning it. And it may be well that you'll need to, do multiple
iterations of this. but will we have a bucket for every single test? I hope
not. That means we're probably doing it Hopefully we end up with about 10
to 15 properties here on the left for the general class of tests that
you're trying to do. hopefully for most of these things just like having a
valid and having a did come back at all is all we really need for let's say
20 tests or so. and then for ones where it's just this test is super
nitpicky. We need to pass multiple different options. It has to be on a
version document. It has to,…

Will Abramson: Cool. Yeah.

Manu Sporny: be in a certain time frame or whatever. we might want to at
that point maybe we have some weird entries in here. Yeah, head well.

Will Abramson: No, that's fair. I mean, I guess like you say, we just got
to do it and explore. And we talked about the first thing that we're going
to do is focus on valid. So, we're going to get this data structure working
or valid. I personally still like the idea of making this data structure be
split up by methods because if you have a look at the report again you
could imagine showing a report where there's a top heading this resolver
name right or maybe resolver a name with the description of a resolver and
then for each DID method there's a subheading and then there's all the test
cases that have been passed against that did method right and it shows you
okay this resolve is really good for this DID method but it's claiming

Will Abramson: to be good for some other DID method but isn't conformant in
X Y and Z places I think that would be really nice information that people
want to learn about different resolvers instead of having this big table
that we scroll to the end and see green check marks like per resolver on
the left it just might be a different way to structure the data right heat.

Manu Sporny: I'm worried that we would then fall into testing dead methods,
which I thought we really didn't want to do. I'm interested to hear on kind
of Joe's take on this. But if people are coming to this thing to figure out
how well a resolver supports a particular DID method, I think that's not
what we're trying to do here. then we should not go down that route, that
is what a be A test suite should have a bunch of resolvers for it and the
resolvers, should exercise every bit of the DID method because they're
going to be things that are very specific to a DID method that we're just
not going to cover in did resolution.
00:45:00

Manu Sporny: So I'm a bit concerned about I don't have any issue with I
think breaking it up by did method here maybe because for example keys from
did web is different from did BTC one right and you're not going to want to
run update and deactivate tests against did key it's just not going to work
right but for DID

Manu Sporny: key or sorry for those tests for activate and deactivate I
wouldn't list did key beside the test bucket because I know it's going to
fail right so I think largely again what we're trying to do here is test
the normative statements in the specification on here should be a normative
statement in the specification and whether or not the implementation is
capable of meeting the normative statement We don't care which did method
it uses to do it. We just care whether or not it passes the normative
statement or not. …

Manu Sporny: hope that helps.

Will Abramson: Thanks. to …

Will Abramson: Benj put his hand down. yeah, no, I think I was going to say
maybe you have convinced me that I think I was going a bit down into the
did methods and really this test what we care about is can a resolver
resolve and pass the resolution thing. so maybe we don't care about the DID
methods at all. It just feels weird to me. But maybe that is the way that
we go. And you just have to provide some dids that demonstrate your
implementation passes all of the resolution spec. But that could be well
and…

Will Abramson: then you across the board from different methods, right?
Marcus could pick a whole handful of different methods and provide
different ones for different test cases. But maybe that's fine. Joe

Joe Andrieu: Yeah, I'm not sure I understood…

Joe Andrieu: what you wanted me to chime in on. Manny, one what I'm hearing
will say is that for BTC1, we have features that the resolver has to
support that we need to test.

Joe Andrieu: So we're going to need to have per did method differences in
terms of the parameters that get sent to the resolver and we need a way to
represent that. And I agree with your general point that what we don't need
to be doing is fleshing out all the ways in which a did method might
respond. But we do have to check that the resolver can actually handle
these additional parameters that the BTC1 needs.

Will Abramson: Thanks Joe man.

Manu Sporny: Yeah, but I was looking for your input on Joe was the idea
that we would become a did method testing group.

Manu Sporny: I think that's out of our charter. I would expect a big
objection from you on …

Joe Andrieu: Yeah, agreed.

Manu Sporny: if I'm like okay.

Joe Andrieu: totally agreed that that's a clearer point. So, yes, I agree.
We should not be testing dead methods.

Manu Sporny:

Manu Sporny: Plus one we have to be able to pass in parameters for
different DID methods so that they can prove that they pass the resolution
for a certain type of resolution feature that they can pass that feature if
they want to support it. U but I don't want it to count as a strike against
a did method…

Joe Andrieu: Yeah.

Manu Sporny: if we're never going to support this feature. We don't believe
in it. We don't want it philosophically we're opposed to it. we don't want
that to count as an X against them because they clearly don't want to
support it.

Will Abramson: Yeah. I was going to say I agree we don't want to be testing
did methods…

Will Abramson: but for me the way we demonstrate clearly we are not testing
is we don't DID methods to provide the responses that they expect a DID to
get resolved to. All our text, all of our tests, And all of our checks on
the resolution results that are retrieved by executing these resolution
requests through the resolver URL that is provided are generic and can be
applied across the board to anything that's resolved. that is the
distinction like we are not checking deep down into the actual results.

Will Abramson: But we still might want to check that for this DID method it
did resolver compatible like it is supporting all the DID resolver features
because you might have a DID resolver that supports a DID method but does
not properly support version for that did method for example and that would
be useful to know
00:50:00

Manu Sporny: I got a little lost at the front of what you were saying.
Will, but agreed with the stuff at the end. maybe if I can just point out
let me bring up VC data model Sweet. H where I hope we end up is as ugly as
this kind of layout is…

Will Abramson: Mhm.

Manu Sporny: but where I hope we end up is here where we have a did
resolution test conformance testing statement that is in the specification
here and whether or not an implementation passes it or not up here like
that that's where I'm hoping we're landing.

Manu Sporny: If we land here where we have a DID method on the left or
categorizing categorization by DID method, I am not supportive of that
because I think that puts us onto a slippery slope of being even conveying
that we're testing DID methods here and we are testing whether or not a
chosen DID method is conformant to the DID resolution things but again We
care about the resolution result in test suite in this specific group. That
I think is what we need to care about. if we put accidentally slip into the
we're testing did methods thing or…

Manu Sporny: even convey that. that's the wrong thing for this group to be
doing. That's it.

Will Abramson: Thanks. …

Will Abramson: Benjamin, we got to close.

Benjamin Young: Yeah, I think that makes the distinction really clear about…

Benjamin Young: what you want to see on the left. I'm not sure it's what
people shopping resolution services want to see on the left. but that's
maybe a different problem.

Will Abramson: Great. Thanks, man.

Manu Sporny: Yeah, that is not our job to help people shopping and it's
really did methods, It's not for resolvers. I think we need to do that
through some other mechanism like we are not supposed to be…

Manu Sporny: what we're supposed to be doing is just testing whether or not
the spec is implementable or not for did resolution. We're not supposed to
be giving people a shopping list on which did methods which did resolvers
they can use. I do agree with you Benjamin people do want that but that is
not our job for this very specific test suite.

Will Abramson: so I think I agree and…

Will Abramson: I'm convinced the only thing that I'm hesitant on is isn't
there a good interoperability argument that we can make if we are showing
look are all these different DID resolvers and look they're conformant
against the DID resolver spec for all these different DID methods and the
tests by which we're demonstrating their conformant are generic and have
been applied to all the DID methods that's pretty valuable but maybe it
isn't what we're trying to get to here kind

Manu Sporny: It is but crawl, walk, run. I think if we can get there, But
we are limited resources, limited time, can we just make it so that we can
get this spec done instead of mean that's great. I do agree with that would
be very powerful and I think if we get a big variation of the did methods
people are using here maybe we can do a post-processing step to show them
show some of these big tech companies that have always said that we don't
actually have interop to show them what every did document that this test
suite came back and…

Manu Sporny: we tested across 10 15 20 different

Manu Sporny: did methods they all had the same format they all met the did
core criteria that is interup right so I think we can make that argument
but let's not create even more work for ourselves and say we're committing
to that just yet I think that'll naturally come out of the test suite let's
see the variation that people use and if there's high variation…

Will Abramson: Mhm.

Manu Sporny: then we can make that point but if there's low variation then
we never signed up for trying to make that point and we can do that, later.

Will Abramson: Okay.

Will Abramson: That's great. I think that, gives me good marching points. I
probably will keep this data structure and explore with Benjamin let's just
get valid working and see if we can get some other implementations
submitting their results to this thing. I mean any final comments before we
close? I mean we'll review this briefly tomorrow right just for five
minutes check with the rest of the group since there's only six of us but I
think this has been a great discussion and it is recorded so that's always
useful yeah…
00:55:00

Manu Sporny: When you shut down make sure to end the call for everyone. And
you might want to stop transcription and stop recording. I want just to
make sure that we top left.

Will Abramson: how do I do that yeah I got it recording stop transcription

Manu Sporny: Yeah. Mhm.
Meeting ended after 00:55:30 👋

*This editable transcript was computer generated and might contain errors.
People can also change the text after it was created.*