RE: [sds-wg] Reminder and Agenda for Confidential Storage Spec Call - Feb 25 2021

RE: Michael has mentioned some other prior-art (in the replication context) that could be reviewed here<https://github.com/decentralized-identity/confidential-storage/issues/162>. I'm sure there's more and across other contexts? Is there other work to document all the prior art that I have missed?

A few weeks ago I highlighted that I was working on a whitepaper to document and diagram the architectural variations for the 4 key layers in a confidential/trust content storage solution. I hope to have a first complete draft “later this week”.  You can find a link to a copy of today’s snapshot at the bottom of this email.

The Preface explains the twists and turns this document has taken and hence, why it has the title that it has…
Context


“Sometimes called “reasoning from first principles,” the idea is to break down complicated problems into basic elements and then reassemble them from the ground up. It's one of the best ways to learn to think for yourself, unlock your creative potential, and move from linear to non-linear results.”

[First Principles: The Building Blocks of True Knowledge(https://fs.blog/2018/04/first-principles/)]

“I think it is most important to reason from first principles rather than by analogy. One of the ways we conduct our lives is we reason by analogy. We do this because something was like something else that was done or it was like what other people were doing. It’s mentally easier to reason by analogy rather than from first principles.”
[First Principles Method Explained by Elon Musk (https://www.youtube.com/watch?v=NV3sBlRgzTI)]

Preface
This is a first-principles software architecture reference model (ARM) whitepaper that documents the comprehensive analysis of the potential architecture variations for the Trust Content Storage Stack (TCS Stack) including detailed interim as well as final assessments and recommendations.

The intent and purpose of this document has taken several strategic turns during the course of its development. Originally entitled Secure Data Storage Working Group (sds-wg) Confidential Storage (CS): Functional Architecture Reference Models (CS-FARMs), the initial goal was to help evolve the SDS Layers diagram in the Confidential Storage 1.0 Specification (https://identity.foundation/confidential-storage/#ecosystem-overview) to make it more useful for discussing the technology and architectural options for supported replication between encrypted data vaults.

This whitepaper evolved quickly over the subsequent weeks from being a few pages in length to more than 200 pages. The whitepaper had forked from its original mission. The purpose of the current version of the whitepaper to fully describe the architecture variations for a new project:  Trusted Content Storage Architecture (TCS Stack).

The work documented here was performed under the auspices of the Trusted Digital Web project in the Hyperonomy Digital Identity Lab of Parallelspace Corporation.


NOTE: The publication of this document coincides with recent discussions (January-February 2021) about secure data storage solutions in the Decentralized Identity Foundation (DIF) Secure Data Storage working group (sds-wg) that were taking place during the development of the Confidential Storage specification. This is not a DIF publication, unofficial, official, or otherwise.

Here’s a commenter link to a copy of today’s snapshot: https://drive.google.com/file/d/1-Ve263KT7svN7oppIHyhK6idXikcbBsn/view?usp=sharing


Any and all feedback is greatly appreciated.

Best regards,
Michael Herman
Sovrin Foundation Self-Sovereignist

Self-Sovereign Blockchain Architect
Trusted Digital Web
Hyperonomy Digital Identity Lab
Parallelspace Corporation

[cid:image003.jpg@01D70FCE.F69FB3E0]



From: sds-wg@lists.identity.foundation <sds-wg@lists.identity.foundation> On Behalf Of Chris
Sent: March 2, 2021 8:38 PM
To: Manu Sporny <msporny@digitalbazaar.com>
Cc: sds-wg@lists.identity.foundation; sds-wg@dif.groups.io; Credentials Community Group <public-credentials@w3.org>
Subject: Re: [sds-wg] Reminder and Agenda for Confidential Storage Spec Call - Feb 25 2021

Thanks Manu for the thoughtful reply.

Yes, this is a next step. Ideally, we'd map every feature in the spec today to
CouchDB documentation. Chris, it would be good for you to volunteer to do this
since you seem to be the one most driven to demonstrate that this is true.
It's something we're going to have to document in time anyway, to demonstrate
why the work needs to be done (or that W3C shouldn't waste their time on it
because... CouchDB exists).
Once you document it (not just the list of requirements we've been going over
during the last month, but all the features in the existing spec as well), we
can review it as a group to see if there is consensus.

Sure, I'm happy to make a first pass on this. Any thoughts on the best format / structure?

Zooming out a bit to look at the big picture... thinking about how SQL was
standardized may help. ... <snip>

Totally agree, standardizing SQL is a perfect example.

To replay your point back to you, but in a different context:

MariaDB (a popular open source relational database) covers many of the common
use case needs for relational databases... so why do we need an SQL standard?

To clarify, I'm not arguing we don't need a standard. To work with the same analogy -- if we're building an SQL standard we should be looking very closely at MariaDB (and other prior art / SQL-like implementations) when developing that standard.

The process of standards are not to innovate (at least, not primarily). It's
to look at everything that's out there and try to standardize the simplest set
of technologies that fit a Pareto distribution... that is, what 20% of
features meet 80% of the use cases. The goal isn't to get the standard to
support 100% of all use cases.

Yes, I agree with that.

I guess one of my concerns relates to feeling like we're looking closely enough at what's out there, but as noted above I'm happy to make a start on that with CouchDB. Michael has mentioned some other prior-art (in the replication context) that could be reviewed here<https://github.com/decentralized-identity/confidential-storage/issues/162>. I'm sure there's more and across other contexts? Is there other work to document all the prior art that I have missed?

The point of meeting 80% of use cases is important, but there's some subtlety in that. Is it 80% of the "total use cases" or 80% of the "most important use cases" -- those are two very different things. And are we dropping use cases because they're "too hard with the current spec implementation" or because "they're not that important"?

Regardless, the ability to have a framework of modularising or extending the capabilities seems like something we should seriously consider. That will minimise the risk of developers ignoring the spec because they're locked in to only working with what's provided out of the box.

> I understand CouchDB is not a specification, but as an implementation it's
> pretty darn close to what we're looking for.

I have a vague concept of what your requirements are, Chris. :) I'm sure you
don't have a solid concept of what Digital Bazaar's requirements are...
or Transmute's... or SecureKey's... or Microsoft's... or Michael's. We're just
scratching the surface, these discussions take a LOOOONG time to get to a
basic understanding on everyone's *public* use cases.

... but the way we get there is to talk about it, and documentation of the
sort you're talking about is vital to that process.

Totally agree :)
_._,_._,_
________________________________
Links:

You receive all messages sent to this group.

View/Reply Online (#86)<https://lists.identity.foundation/g/sds-wg/message/86> | Reply To Group<mailto:sds-wg@lists.identity.foundation?subject=Re:%20Re%3A%20%5Bsds-wg%5D%20Reminder%20and%20Agenda%20for%20Confidential%20Storage%20Spec%20Call%20-%20Feb%2025%202021> | Reply To Sender<mailto:chris@verida.io?subject=Private:%20Re:%20Re%3A%20%5Bsds-wg%5D%20Reminder%20and%20Agenda%20for%20Confidential%20Storage%20Spec%20Call%20-%20Feb%2025%202021> | Mute This Topic<https://lists.identity.foundation/mt/80910574/1997675> | New Topic<https://lists.identity.foundation/g/sds-wg/post>
Your Subscription<https://lists.identity.foundation/g/sds-wg/editsub/1997675> | Contact Group Owner<mailto:sds-wg+owner@lists.identity.foundation> | Unsubscribe<https://lists.identity.foundation/g/sds-wg/leave/9912086/1997675/2030013897/xyzzy> [mwherman@parallelspace.net]
_._,_._,_