W3C home > Mailing lists > Public > public-credentials@w3.org > August 2019

Re: Datashards and Secure Data Hubs... how to achieve convergence?

From: Christopher Lemmer Webber <cwebber@dustycloud.org>
Date: Sat, 17 Aug 2019 12:07:45 -0400
To: Adrian Gropper <agropper@healthurl.com>
Cc: Serge Wroclawski <serge@wroclawski.org>, public-credentials@w3.org
Message-ID: <87zhk7wzam.fsf@dustycloud.org>
Hi Adrian!  :)

Adrian Gropper writes:

> Let’s consider the prescription use-case we maintain as part of the HIE of
> One self-sovereign tech stack. We need to:
> - control and share signed data
> - encrypt the data as much as practical
> - all parties keep archival copies in case of dispute
> - revoke the data (prescription) when an inspector decides (dispenses).

Yes.  And I'll notice that this supports all of that; it's even possible
for some groups which are holding data to not actually know what the
data is that they're holding; in that sense Datashards and Secure Data
Hubs are very complementary, except that Datashards provides a URI that
we can refer to things by.

> Would we code the prescription into a Verifiable Credential or a Datashard
> or a combination?

A datashard is really a data-storage-mechanism and URI type.  We'd still
have all the usual data, as JSON-LD, in a Verifiable Credential; we'd
just save that Verifiable Credential as a datashards object and upload
it somewhere.  So say we uploaded:

     "@context": [
      "type": ["VerifiableCredential", "PrescriptionCredential"],
      "issuer": "https://example.edu",
      "issuanceDate": "2010-01-01T19:73:24Z",
      "credentialSubject": {
        "id": "did:example:ebfeb1f712ebc6f1c276e12ec21",
        "prescription": {....}
      "revocation": {
        "id": "http://example.gov/revocations/738",
        "type": "SimpleRevocationList2017"
      "proof": {....}

Now we have two choices.  We can upload this as an immutable datashards

  $ raco idsc --upload /tmp/vc-example.txt

That idsc is now the URI above for this object.

Now if we run:

  $ raco idsc --get idsc:0p.clfCj_4cJa24pymJDnxE-REJETxTDktJRQAAdLpZr3A.o06THIeG1jUr0S62rVQBH0gsIIcIgaoSRcF5DnPDXnI

... we can get it back.

We can then use this as the id of the object, refer to it in other
documents, like so:

  {"@context": [
   "type": "FulfillPrescription",
   "prescription": "idsc:0p.clfCj_4cJa24pymJDnxE-REJETxTDktJRQAAdLpZr3A.o06THIeG1jUr0S62rVQBH0gsIIcIgaoSRcF5DnPDXnI"}

(I realize that this is an incomplete example, I'm just showing it for

However, this is "immutable", ie, unchanging.  We could also choose to
upload it via a "mutable" datashards object, and maybe get back a URI like:


We could store the same data there.  However, unlike the idsc: version,
we can update this one... we could even put the revocation directly on
the VC if we prefer.  However, mdsc: objects have some challenges with
synchronizing objects that idsc objects do not.  We can worry about that
later though.

> Let’s assume the doctor (issuer) uses her DID wallet to:
> - hold a credential acceptable to the pharmacist (inspector1), and
> - to sign the prescription in an accountable way (inspector2).
> The patient wants to shop the prescription to various pharmacies (based on
> their public pricing), keep it private unless a dispute arises, and have
> the flexibility of transacting with the pharmacist online or in-person.
> What’s the preferred way to manage routing to a particular pharmacy and
> then revocation if and when a pharmacist dispenses the prescription? Is the
> solution different for VC vs. Datashard?
> Adrian

There's no VC vs Datashard; they're completely complementary!  (As are
Datashards and Secure Data Hubs.)  Here's a vision of how they can work

 - Save VCs as Datashards objects.
 - Upload them to Secure Data Hubs.
 - Even though the different "chunks"/shards of the Datashards Object
   can be saved in various places (both archived at the pharmacy and
   also on your Secure Data Hub).
 - When you give the pharmacist the idsc: or mdsc: URI, at that point
   they can read the data.  Until they have that, they might even be
   able to "hold" the data, but they can't read it or know what it's
 - No matter whether it's being held by the pharmacist's local database
   or at your Secure Data Hub, there's the convenience that the
   Datashards URI refers to the same data in both places!

Does that answer your question, Adrian?  I hope you can see that all of
these systems work together beautifully!

 - Chris

> On Sat, Aug 17, 2019 at 9:37 AM Christopher Lemmer Webber <
> cwebber@dustycloud.org> wrote:
>> As promised, here's that rant.  It's not actually all that ranty.  ;)
>> So having seen Datashards and DB's work on Secure Data Shards, and
>> knowing that I occasionally contract for DB, you might think that maybe
>> DB also had something to do with Datashards.  That's not the case, other
>> than that I've been trying to convince them of the ideas for the last
>> few years (we do talk a lot, but I haven't been paid for any work at all
>> in this area).  Over time we've come closer, but our work has happened
>> seperately, and we keep checking in and seeing that we come to many of
>> the same conclusions for different reasons, but not all of them.  So the
>> ideas are close, but they aren't yet aligned.  I hope at RWoT we can get
>> some better convergence.
>> There is one major difference that, in truth, barely matters (except
>> that it kind of does):
>>  - A difference in how data is serialized.  We can discuss this, it's
>>    not too important though.  It's easy enough to get convergence on
>>    this issue.
>> There are also several things that I used to argue for that we've
>> already come to convergence on:
>>  - The need for symmetric encryption on content-addressed content
>>    (well, at least we agree it's needed for some things; I'll argue
>>    for all CAS things later)
>>  - The need to chunk content into 32kb chunks
>> Yay, convergence!
>> Then there are several different differences which actually matter:
>>  - Datashards not only specifies an immutable system, but also a mutable
>>    system that layers on top of the immutable data.  The design is
>>    mostly taken from Tahoe-LAFS (which has done almost everything right
>>    except for making their system ~universal in a way that can be
>>    applicable on the web... in Tahoe you have to run things on specific
>>    clusters).
>>  - Datashards has two universal URI schemas that can exist *outside* of
>>    a data hub.  idsc: is for immutable data, mdsc: is for mutable data.
>>    This is, I would argue, a nice feature.  You can use these for
>>    everything you would normally put an https: link; this includes
>>    json-ld contexts and linking to cat photos and all sorts of things.
>>    (This is what I mean by Datashards building "universal web
>>    primitives".)
>>  - Datashards can exist *anywhere*.  You can put them on a usb key, you
>>    can stick them in a key-value web store, whatever.  I'm arguing that
>>    Secure Data Hubs are a good idea, except that they should just be
>>    "a common web interface to store your Datashards".  That's a big
>>    feature because then you have a URI so you can refer to those same
>>    objects univerally, the same way you an refer to an http uri, etc.
>>  - I think DB isn't yet convinced that we need symmetric encryption for
>>    *all* content-addressed content.  Again, I refer you to the podcast
>>    for why this is, but I think that content-addressed peer to peer
>>    systems that don't do symmetric encryption and chunking by default
>>    not only don't have a nice privacy layer by default, but they put
>>    node operators at risk.  (Imagine if you worked for a volunteer
>>    postal delivery service, but instead of packages and letters being
>>    wrapped in opaque paper, they were wrapped in syran wrap.  You might
>>    start to feel like you need to make moral judgements for what you're
>>    delivering, and even worse, making moral judgements may become so
>>    commonplace that eventually a law is enacted telling all postal
>>    service volunteers that they *must* examine contents before
>>    delivering them, and are likewise liable for their contents.
>>    Sometimes it's better to know less.)
>> We disagree on the four above points somewhat, but I'll argue that it's
>> really little to no extra cost (it may even be less) to switch the
>> Secure Data Hubs approach to the way I'm describing (being *a* place to
>> store Datashards as opposed to being content which can only live at that
>> node and can't be conveniently referenced outside of it), and the gains
>> are very great.
>> But, RWoT is a nice place for us to work on resolving these differences,
>> which I am convinced are really not so large, but the benefits of
>> resolving them are very high.  I look forward to working through them at
>> RWoT. :)
>>  - Chris
>> Christopher Lemmer Webber writes:
>> > Since I intend to bring up both in a significant way at Rebooting Web of
>> > Trust, it may be useful to hear that we put out two episodes on the
>> > podcast I co-host: one about Datashards:
>> >
>> >   https://librelounge.org/episodes/26-announcing-datashards.html
>> >
>> > and one about OcapPub:
>> >
>> >
>> https://librelounge.org/episodes/25-managing-spam-and-hate-speech-on-the-fediverse.html
>> >
>> > More about Datashards:
>> >
>> https://github.com/WebOfTrustInfo/rwot9-prague/blob/master/topics-and-advance-readings/datashards-rationale.md
>> >
>> > More about OcapPub:
>> > https://gitlab.com/spritely/ocappub/blob/master/README.org
>> >
>> > You may wonder how Datashards and Secure Data Hubs can fit together.
>> > For that, I will follow up to this email with a rant. ;)
>> --
> Adrian Gropper MD
> HELP us fight for the right to control personal health data.
> DONATE: https://patientprivacyrights.org/donate-3/
Received on Saturday, 17 August 2019 16:08:09 UTC

This archive was generated by hypermail 2.4.0 : Thursday, 24 March 2022 20:24:55 UTC