Reference implementation anti-pattern (was: Re: New Work Item Proposal: Universal Wallet 2020)

On 6/24/20 4:45 PM, Wayne Chang wrote:
> Manu, could you please elaborate on your concerns about calling
> certain code a "reference implementation"? What drawbacks are there,
> and what else could we call it? "Example implementation"? "Conforming
> implementation"?

Thanks to Leonard, DanB, Dmitri, Kim, and DanielH's comments, which do a
much better job than I would have elaborating on why using the term
"reference implementation" is problematic.

The biggest issue is the "gold standard" language in the Wikipedia
definition, which I don't disagree with.

The problem that we've seen play out time and time again in the global
standards process over the last 25+ years is that "reference
implementation" for algorithms work out well. Reference implementations
for non-trivially complex systems, like what is being proposed in
Universal Wallet 2020, end up with software that is:

1. Not maintained since it's often unfunded work.
2. Not suitable for production use (see #1 above), and thus is
   dangerous if developers use it as a reference.
3. Politically unstable because of the power dynamics -
   developers think that features implemented in the
   reference implementation are blessed by the group, when
   it may be that the developer that did the reference
   implementation just through the optional feature was neat.
   To combat this, committees form around reference
   implementations and suck energy away from the spec.
4. Always behind the state of the market, because of #1 above
   and the aforementioned need to "design by committee".

Fundamentally, it was the failure of reference implementations that led
to conformance test suites and ensuring that the specification was the
gold standard (and not some piece of software).

To put it another way, reference implementations for non-algorithmic
standards have been an anti-pattern for a very long time. You'll note
that many of the folks speaking against it have been around the block
several times in the standards space -- they are speaking from
experience, possibly having proposed the same thing in the past and
having it blow up in their face. I know that's what's happened to me in
every reference implementation attempt I've participated in. :)

So, what do we call it? An implementation... because that's all it is.
If it's good, it's a conforming implementation... hopefully, one of many.

-- manu

PS: Also, I'll note that this was far down on our list of concerns. We
don't need to bikeshed it now. :)

-- 
Manu Sporny - https://www.linkedin.com/in/manusporny/
Founder/CEO - Digital Bazaar, Inc.
blog: Veres One Decentralized Identifier Blockchain Launches
https://tinyurl.com/veres-one-launches

Received on Thursday, 25 June 2020 13:24:11 UTC