Re: A simpler approach to DID services and content-addressing

On Thu, Apr 4, 2019 at 8:13 AM Michael Herman (Parallelspace) <
mwherman@parallelspace.net> wrote:

> Nice.  It’s actually very good to see some convergence :-)
>

Yes.


>
> *RE: “uses more delimiter characters”*
>
>
>
> Drummond , this is neither an accurate or fair observation/comment.
>

No offense meant. See below.


>
>
> Aside from, “=”,  “””” and “’” which are commonly used to bracket option
> values today, the Hyperonomy *did-uri-spec* grammar only uses two (2)
> special characters:
>
>    1. the TRANSFORM (“!”) character and
>    2. the OPTION (“$”) character.
>
> So the reason for the observation/comment is that, while those additional
characters are understandable from a programming language or development
perspective, they add "weight" to a URL spec that we are trying to keep as
simple and lightweight as possible. In my discussions with Markus, John
Jordan and Stephen Curran, we believe the matrix parameter approach
provides the same functionality with fewer characters. But if we're wrong
about that, please educate us.



> *Leveraging existing, mature, international standards and grammars*
>
>
>
> A key attribute of the *did-uri-spec* grammar is that it leverages (will
> literally embed significant portions of) the existing, mature,
> international OASIS/ISO OData (Open Data) standard:
>
>
>
>    -
> *OData Version 4.01. Part 2: URL Conventions *
>    http://docs.oasis-open.org/odata/odata/v4.01/cs01/part2-url-conventions/odata-v4.01-cs01-part2-url-conventions.html#_Toc505773216
>
>
>
>    -
> *OData ABNF Construction Rules Version 4.01 *
>    http://docs.oasis-open.org/odata/odata/v4.01/cs01/abnf/odata-abnf-construction-rules.txt
>
> First, I do appreciate the references to existing formats. Second, I
appreciate that OData uses this grammar and parameter patterns. However in
looking over those specs, I have the same reaction, which is that they are
heavierweight that what we need at the DID layer. IMHO OData is designed
for a much broader set of data transfer use cases than we are trying to
tackle with DIDs.

What do others think?



> *Principles-based Architecture and Design*
>
>
>
> Second lastly, the one thing I’m not seeing in the other proposals is a
> defined set of principles that are being used to drive the alternative
> proposals.
>

This is a great point, and important enough that I added it to the agenda
for today's DID Spec call
<https://docs.google.com/document/d/1qYBaXQMUoB86Alquu7WBtWOxsS8SMhp1fioYKEGCabE/edit#heading=h.jgso2ldlbwy>
(1PM
PT—call-in details on the meeting page
<https://docs.google.com/document/d/1qYBaXQMUoB86Alquu7WBtWOxsS8SMhp1fioYKEGCabE/edit#heading=h.e7z6ck4q5qp4>
).


> For example,
>
>    - is readability an important principle?
>
> Great question. My own answer is, "Yes, at least as far as the syntax does
not put off developers."

>
>    -
>    - is compatibility with and leveraging of existing standards an
>    important principle? …if so, which standards? etc. etc.
>
> See below. Other than RFC 3986, I think any other standard for URI
construction is informative, not normative.


>
>    -
>    - is the #OpenToInnovation principle important/a pre-requisite?
>
> I translate that as "is it extensible?". My answer is yes. Curious to know
about others.


>
>    -
>    - is RFC 3986 compatibility a required principle?
>
> Yes. Period.


>
>    -
>    - “how big is the tent” from a *did-uri* grammar perspective?  …for
>    example, does the proposal support transform options at the *did-root*
>    and *did-method* as well as *did* levels in the grammar?
>
>
As far as the DID spec is concerned, my default answer is "only at the
post-DID level". However I could see the DID Resolution spec having a
superset grammar that opens up the use of parameters at the higher levels.


>
>    -
>    - …many other principles?
>
> Examples?



> In my experience, without a firm set of agreed upon underlying principles
> defined upfront, conversations tend to go around and around in circles
> …ending with a weak consensus or, worst case, no useful consensus.
>

My experience is not quite as bad, but I do agree that design principles
are a good thing.


>
>
> *English Language or Assembly Language?*
>
>
>
> Lastly, I can’t help it but whenever I see grammars that depend on the “;”
> a lot, the examples tend to look like some sort of low-level,
> assembly-language machine instructions.
>
>
>
> That’s my 2 cents Canadian. :-)
>

In my work with ABNF, I find that every person has their own option of how
it "reads" based on their linguistic, cultural, and technical backgrounds
and experiences. So IMHO when it comes to aethetics, all we can do is try
to seek the consensus of the CCG.

Best,

=Drummond


>
>
> Best regards,
>
> Michael Herman (Toronto/Calgary/Seattle)
>
> Independent Blockchain Developer
>
> Hyperonomy Business Blockchain / Parallelspace Corporation
>
>
>
> W: http://hyperonomy.com
>
> C:  +1 416 524-7702
>
>
>
>
>
> *From:* =Drummond Reed <drummond.reed@evernym.com>
> *Sent:* April 4, 2019 3:44 AM
> *To:* W3C Credentials CG <public-credentials@w3.org>
> *Cc:* jonnycrunch <jonnycrunch@me.com>; Manu Sporny <
> msporny@digitalbazaar.com>; Michael Herman (Parallelspace) <
> mwherman@parallelspace.net>
> *Subject:* Re: A simpler approach to DID services and content-addressing
>
>
>
> My apologies for joining this thread late. I flew up from Seattle to
> visit John Jordan and Stephen Curran and their team at BC Gov yesterday.
> One benefit of the visit was that it gave us a long whiteboard session to
> go over both Manu's and Michael Herman's proposals. We were also able to
> get Markus on a Zoom to get his thoughts.
>
>
>
> The outcome was the proposal below, called the "matrix parameters syntax"
> proposal thanks to Markus' web research on URI parameter syntax (it turns
> out no less than Tim Berners-Lee proposed matrix parameter syntax in 1996
> <https://www.w3.org/DesignIssues/MatrixURIs.html>).
>
>
>
> We (the BC Gov team, Markus, and I) offer this up for discussion here and
> on the DID Spec meeting tomorrow (Thursday 1PM PT—see reminder email to the
> list in the morning) as a "middle way" between the ultra lightweight
> approach propose by Manu and the heavier syntax proposed by Michael. Our
> conclusions were:
>
>    1. The issue we saw with Manu's proposal is that there is no syntactic
>    delimiter between the "naked DID" (that identifies the DID subject) and the
>    parameters.
>    2. The only issue we had with Michael Herman's proposal is that the
>    syntax seemed heavier-weight (i.e., uses more delimiter characters) than
>    necessary, particularly for a URL grammar.
>
> When Markus found the matrix parameter syntax, it turns out to use the
> semicolon that has long been the proposed delimiter to separate the "magic
> part" (as Manu puts it) from the naked DID. And it allows a sequence of
> parameters just like the naked DID allows a sequence of colon segments.
>
>
>
> Here is a link to a Google doc of the proposal
> <https://docs.google.com/document/d/1TctFY8euBH2wq7Z8c9KccICDZUGZplvhoqlHlFMahGk/edit?usp=sharing>.
> A copy is included below for easy reference and archiving on the CCG list.
>
> *DID Matrix Parameters Syntax Proposal*
>
> 2019-04-04
> Introduction
>
> This proposal describes a single simple flexible mechanism to enable DIDs
> to be extended to address other resources besides a DID document (both
> immutable/decentralized resources and mutable Web resources) while keeping
> all the advantages of DIDs and DID documents (global uniqueness,
> persistence, immutability, verifiability, decentralization).
> Matrix Parameters
>
> This proposal uses a syntax for adding parameters to a URI originally
> proposed by Tim Berners-Lee
> <https://www.w3.org/DesignIssues/MatrixURIs.html> and others in 1996, and
> more recently discussed in this Stack Overflow thread
> <https://stackoverflow.com/questions/2048121/url-matrix-parameters-vs-request-parameters>.
> Each matrix parameter is delimited with a semicolon, and the name of the
> parameter is separated from the value by an equals sign.
> ANBF
>
> The revisions required to support matrix parameters in DID URLs are
> highlighted in *red* below. Note that matrix parameters are all expressed
> within the authority component of a DID URL, leaving the path, query, and
> fragment for normal Web URL usages.
>
> *did                = "did:" method ":" method-specific-id*
>
> *method             = 1*method-char*
>
> *method-char        = %x61-7A / DIGIT*
>
> *method-specific-id = idstring *( ":" idstring )*
>
> *idstring           = 1*idchar*
>
> *idchar             = ALPHA / DIGIT / "." / "-"*
>
> *did-url            = did ***( ";" param )*
> *path-abempty                     [ "?" query ] [ "#" fragment ]*
>
> *param              = param-name [ "=" param-value ]*
>
> *param-name         = 1*param-char*
>
> *param-value        = *param-char*
>
> *param-char         = ALPHA / DIGIT / "." / "-" / "_" / pct-encoded*
> Parameter Names
>
> The ABNF above does not specify any particular parameter names. The
> proposal is for the DID spec to specify several standard parameter names
> for use with addressing secondary resources within a DID document, and then
> leave other parameter names to be specified by DID method specifications
> (some of which could, with broad enough adoption, migrate to become
> standard parameter names in future versions of the DID spec).
> Basic Examples
>
> These examples illustrate possible parameter names. VDR = Verifiable Data
> Registry.
>
> *Example*
>
> *Description*
>
> *did:example:1234abcd;service=agent*
>
> Selects a service by name
>
> *did:example:1234abcd;service-type=hub*
>
> Selects a service by type
>
> *did:example:1234abcd;type=schema;id=1234*
>
> Identifies VDR content by type
>
> *did:example:1234abcd;type=creddef;id=a1bc3d4*
>
> Identifies VDR content by type
>
> *did:example:1234abcd;hashlink=**hl:*
>
> *zQmWvQxTqbG2Z9HPJgG57jjwR154cKhbtJenbyYTWkjgF3e:*
>
> *zuh8iaLobXC8g9tfma1CSTtYBakXeSTkHrYA5hmD4F7dCLw8XYwZ1GWyJ3zwF*
>
> Identifies VDR content using a standard content-addressing syntax (
> hashlink <https://tools.ietf.org/html/draft-sporny-hashlink-00>)
>
>
> Service Reference Example
>
> A DID service reference is a DID URL that includes a path and/or query
> (and optionally a fragment after the path and/or query). The proposal is
> that the DID Resolution spec would define an unambiguous transform from a
> DID service reference into a concrete URL.
>
> *did:example:1234abcd;service=agent/path/goes/here?forward=true#alert*
>
> If the selected service endpoint (whose id property had a value of
> “agent”) had the following URL…
>
> *https://example.com/12345678 <https://example.com/12345678>*
>
> Then the transform from the original DID service reference would produce
> the following concrete URL:
>
>             *https://example.com/12345678/path/goes/here?forward=true#alert
> <https://example.com/12345678/path/goes/here?forward=true#alert>*
>
>
>
>
>
>
>
> On Wed, Apr 3, 2019 at 7:30 AM Michael Herman (Parallelspace) <
> mwherman@parallelspace.net> wrote:
>
> Correction: “Document Documents” should have read “DID Documents”
>
>
>
> *From:* Michael Herman (Parallelspace) <mwherman@parallelspace.net>
> *Sent:* April 3, 2019 8:26 AM
> *To:* jonnycrunch <jonnycrunch@me.com>
> *Cc:* Manu Sporny <msporny@digitalbazaar.com>; W3C Credentials CG <
> public-credentials@w3.org>
> *Subject:* RE: A simpler approach to DID services and content-addressing
>
>
>
> RE: - push the ABNF URL rules to be declared in the individual DID
> Methods, much like how currently RESTful API document their endpoints
>
>
>
> The *did-uri-spec* fully supports this requirement via something called
> Domain-Specific DID Grammars. For example, there is a DSDG for “Document
> Documents”; another for “DID Document Collections”, etc. (and they can be
> composed).  Watch this video to learn more (slightly dated already):
> https://www.youtube.com/watch?v=IdLm2jHuADg&list=PLU-rWqHm5p45c9jFftlYcr4XIWcZb0yCv&index=4
>
>
>
> Also note that except for the Domain-Specific DID Grammar concept, the
> *did-uri-spec* grammar is strictly a syntax-level specification.  It
> doesn’t mandate the implementation of any predefined set of
> *$transformer-options*.  It’s just a set of generic syntax rules for a
> generic *did-uri* syntax.  I’ve include a copy below.  NOTE: This is
> different from the “DID ABNF” grammar where a strong set of semantics is
> implied by encoding of the special characters “#”, “!”, and “$”.
>
>
>
> [In a few minutes, I’ll be starting a roadtrip to Seattle for the next
> couple days and won’t be online as much.  But feel free to call me in the
> car – no advance notice required – I’ll welcome the conversation.  +1
> 416-524-7702]
>
>
>
> Best regards,
>
> Michael
>
>
>
>
>
> Best regards,
>
> Michael Herman (Toronto/Calgary/Seattle)
>
> Independent Blockchain Developer
>
> Hyperonomy Business Blockchain / Parallelspace Corporation
>
>
>
> W: http://hyperonomy.com
>
> C:  +1 416 524-7702
>
>
>
>
>
> *From:* jonnycrunch <jonnycrunch@me.com>
> *Sent:* April 3, 2019 7:52 AM
> *To:* Michael Herman (Parallelspace) <mwherman@parallelspace.net>
> *Cc:* Manu Sporny <msporny@digitalbazaar.com>; W3C Credentials CG <
> public-credentials@w3.org>
> *Subject:* Re: A simpler approach to DID services and content-addressing
>
>
>
>
>
>
>
> Both of these approached seem a bit over-engineered and over-reaching for
> my taste.
>
>
>
> I really like the content id and hash-linking approach, and can work some
> magic to transform and interoperate.  however….
>
>
>
> My big concern regarding this concept of all DID methods using the same
> ABNF URL rules to all for “Changing service providers” as that not all
> service providers will support all methods.
>
>
>
> Not to mention in our approach (IPID) there is no service provider.  You
> are your own service provider.  It is a distributed, not just
> decentralized, solution.
>
>
>
> This seems to be overreaching authority into each of the DID methods and
> forcing compliance and thus losing autonomy and self-sovereignty
>
>
>
>
>
> I’d like to prioritize:
>
> - ABNF rules to separate the naked DID and rest fo the DID URL
>
> - push the ABNF URL rules to be declared in the individual DID Methods,
> much like how currently RESTful API document their endpoints
>
> - finalizing the DID document spec and clarify the required attributes
> with hardened rules for a JSON schema declaration and testing
>
> - work on interoperability tests for resolution and validating
> signatures/proofs issued as verifiable credentials
>
>
>
>
>
> Best,
>
>
>
> Jonny
>
>
>
>
>
>
>
>
>
> On Apr 3, 2019, at 12:49 AM, Michael Herman (Parallelspace) <
> mwherman@parallelspace.net> wrote:
>
>
>
> Manu,
>
>
>
> Here's simpler solution using the Hyperonomy *did-uri-spec* specification
> and grammar.  See the attached "Page 47" slide where I've tried to:
>
>    1. capture your 3 use cases, then
>    2. create a *did-uri-spec* compliant an example *did-uri* for each use
>    case, and finally
>    3. added a description of the output.
>
> I've also attached a copy of the *did-uri-spec* grammar (defined using
> ABNF notation).  See attached image with a black background.
>
>
>
> RE: "resolution algorithms"
>
>
>
> This is a little more difficult to explain but for the most part, the
> "resolution algorithms" you describe are actually "parsing algorithms".
> When using a grammar defined using ABNF notation, the pattern matching,
> etc. that takes place in the automatically generated parser (automatically
> generated from the ABNF description of the grammar) as it processes a
> *did-uri* is automatic. i.e. there's not need to describe it English if
> you have a description using ABNF notation.  I hope this makes sense.
>
>
>
> Best regards,
>
> Michael
>
>
>
>
>
>
>
>
> ------------------------------
>
> *From:* Manu Sporny <msporny@digitalbazaar.com>
> *Sent:* March 31, 2019 1:38 PM
> *To:* W3C Credentials CG
> *Subject:* A simpler approach to DID services and content-addressing
>
>
>
> Apologies for missing the last DID spec call, some of us thought it had
> been cancelled due to KNOW2019 and travel. I was reading the minutes
> from the last meeting[1] and was happy to see two use cases identified
> as driving factors for the DID URI scheme syntax (aka, ABNF) discussion.
> I'm also attempting to build on the work that Drummond and Ken did
> during RWoT8 (so you two may see some stuff that's familiar in here).
>
> Let me try and summarize the use cases that seem to be driving the DID
> URI scheme syntax:
>
> 1. Pat wants to publish blogging content in a way where they can switch
>    service providers, but the relative URLs don't change. For example,
>    <PORTABLE_SERVICE_PROVIDER_URL>/2019-03-01/lunch-in-barcelona
>    ... which would be transformed to:
>    <CURRENT_SERVICE_PROVIDER_URL>/2019-03-01/lunch-in-barcelona
>
> 2. Yael wants to publish content in a way that ensures the content
>    integrity of the content. For example,
>    <CONTENT_URL><SEPARATOR><CONTENT_INTEGRITY_CHECK>
>    ... which would be transformed to:
>    <PROTECTED_CONTENT_URL> (which returns tamper-evident data)
>
> Now, let's narrow that down to the specification we're talking about,
> which is the DID Specification. The pattern would change to something like:
>
> <START_OF_DID><MAGIC><PATH>
>
> and
>
> <START_OF_DID><MAGIC><CONTENT_INTEGRITY_CHECK>
>
> Much of this discussion has been around the <MAGIC> bit above. We know
> what we want at the end, but there are many things that could go in the
> middle. I should also note that these are two totally different use
> cases and the group is probably thrashing because you're attempting to
> solve both of them simultaneously. In this case, though, I think there
> is at least one simple answer that doesn't require overly-complicated
> microsyntaxes. So, here goes:
>
> Use a colon-delimited keyword that you tack on to the end of a "bare DID".
>
> That's it. Here's how it looks in practice for the two use cases above.
>
> For the "Portable URL":
>
> did:example:12345678:path:blog:/2019-03-01/lunch-in-barcelona
>
> ... which would be transformed to:
>
> https://example.org/2019-03-01/lunch-in-barcelona
>
> For the "DID with Content-addressing" (example assumes a Sovrin-like
> requirement to get a content addressed schema):
>
> did:example:schema:hl:z3aq31uzgnZBuWNzUB
>
> ... which would enable you to fetch things from that particular DID
> Registry using content-based addressing.
>
> The general matching pattern for the syntaxes are:
>
> For the "Portable URL" use case:
>
> did:<did-method>:<method-specific-id>:path:<service-id>:<service-path>
>
> and for the "DID with Content-addressing" use case:
>
> did:<did-method>:<method-specific-id>:<hashlink>
>
> Note how the use cases are handled with things that you put *at the end*
> of the DID URI syntax? This is on purpose for two reasons:
>
> 1. It ensures that DID Method authors have very broad control over what
>    happens in <method-specific-id>, and
> 2. It enables decentralized innovation to occur while providing a clear
>    adoption path into the core DID spec. (I think this is a nuance that
>    many people might not get right now, but that's ok).
>
> Here are two potential resolution algorithms for both use cases:
>
> For the "Portable URL" use case:
>
> 1. Search for ":path:" from the end of the DID URL.
> 2. Split on the first two colons, you should have a 3-tuple:
>    ("path", service-id, service-path).
> 3. Search the DID Document "service" property for a service
>    with an "id" ending in "#<service-id" and save this value
>    as "service-prefix".
> 4. Concatenate "service-prefix" with "service-path" and return
>    this value.
>
> For the "DID with Content-addressing" use case:
>
> 1. Search for ":hl:" from the end of the DID URL.
> 2. Split on the first colon, you should have a 2-tuple:
>    ("hl", hashlink-value).
> 3. Dereference the DID URL and cryptographically hash
>    the content.
> 4. Compare the hash of the content against the value
>    of the hashlink. If the hashes match, the content is
>    secure.
>
> I believe the proposal above addresses all of the use cases raised by
> Evernym and Sovrin. I also think it's compatible with Veres One and
> IPFS-based DID Methods. The benefits of this approach are:
>
> 1. It ensures that DID Method authors have very broad control over what
>    happens in their <method-specific-id>, and
> 2. It enables decentralized innovation to occur for these sorts of
>    DID URI syntax extensions while providing a clear adoption path into
>    the core DID spec, and
> 3. It doesn't require microsyntaxes more than what we have in the
>    specification right now, and
> 4. The grammar parsing rules are extremely simple, and
> 5. It only requires one type of separator character for the DID URI
>    syntax, the ":" character.
>
> What did I miss? Why doesn't this work? I checked this against all the
> current use cases (but did not elaborate on every one as this email is
> long enough as it is). Would this work for your use case?
>
> -- manu
>
> [1]
> https://raw.githubusercontent.com/w3c-ccg/meetings/gh-pages/2019-03-28-did-spec/2019-03-28-irc.log
>
> [2]
> https://github.com/WebOfTrustInfo/rwot8-barcelona/blob/master/draft-documents/did-spec-refinement.md#feature-refinement
>
> --
> Manu Sporny (skype: msporny, twitter: manusporny)
> Founder/CEO - Digital Bazaar, Inc.
> blog: Veres One Decentralized Identifier Blockchain Launches
> https://tinyurl.com/veres-one-launches
>
> <manu-page-47.png><did-uri-spec-simple-2019-04-03.abnf.png>
>
>
>
>