Re: Why use DER rather than BER?

On Wed, Mar 30, 2016 at 5:55 PM, Jim Schaad <ietf@augustcellars.com> wrote:

> I was not around when this decision was made, but I am curious why it was
> decided that we should do DER encoding and decoding rather than the more
> natural BER.  Since DER is a proper subset of BER, the statement that you
> need to have both a DER and a BER decoder seems to be wrong.  A BER decoder
> would successfully decode DER with no problems.  I do not know of any
> security reasons why the document should prefer DER to BER.  It is not like
> we care that there is a single encoding for a specific value or not.  We
> are
> not signing the output at all.  The current requirement of having DER
> decoders means that there are going to be some private keys that were
> exported from a random source that will not import into the WebCrypto world
> since they are BER encoded (as per the spec) and thus cannot be
> successfully
> parsed.
>
> I don't necessarily want to change the decisions, but it would seem that
> much of Ryan's objections disappear if we allow for BER decoders.  This
> would not make any requirements on encoders - they could still be required
> to be DER if it was desired.
>
> Jim
>
>
I... struggle a lot in how best to reply to this mail, Jim, because I would
hope it would be understandably obvious why one would be preferable,
especially given your involvement in the security space for as long as you
have been. If the tone in this email comes off wrong, I'd like to
premptively apologize, but I an admittedly quite boggled as to how the
question came to be.

You're correct that a BER decoder can (or should) be able to decode DER,
although the converse is not true - a DER decoder cannot decode BER.

As such, to support BER, it would either be required that an implementation
contains:
1) A BER encoder and BER decoder
2) A DER encoder and a BER decoder

Now, I fundamentally disagree with the acceptance that "there are going to
be some private keys that were exported from a random source that will not
import into the WebCrypto world" as being a valid use case. There are many
awful encoders, as you mentioned in respect to certificates, and from a
Chrome implementation side, there is absolutely zero interest or plans to
accept that as a justification to design a new system with bugs and
duct-tape. We (Chrome) are actively working to remove the acceptance of
such certificates and keys in other spaces. As it relates to other systems,
quite simply, my (personal) view is that if you have such systems, they
should adapt or perish. You don't get to use WebCrypto with your awful,
broken encoder. This is fundamentally no different than the decision not to
support MD5 or 3DES in the API - just because you have a legacy system that
expects MD5/3DES is not sufficient justification in-and-of-itself to
support, and just because you have a broken system spitting out improperly
encoded (or BER encoded, which I'm building to, I promise) doesn't mean
that it's a justification in-and-of-itself to support. As mentioned on the
other thread, if you need to massage your data, you can use Javascript on
the client side or filter it on the server side before sending. This
position, I'll note, is wholly aligned with the Extensible Web Manifesto -
we give the building blocks, but there's no intrinsic reason why the
browser needs to handle "Awful Legacy X", especially if you can polyfill
(securely) in JS.

But that's a bit of a side-show (though related) to your question of "Why
not accept BER decoders". The simple answer is because BER is abyssmal. It
is a pain to implement and I would go as far to say as it CANNOT safely be
done while adhering to modern software design.

Consider that BER allows for indefinite length encodings, for example. One
can construct an encoding that does not terminate. You might not understand
why that's a concern, given the current BufferSource API, but that's
fundamentally inconsistent with any intent to offer a streaming API. This
problem is very similar to the notion of "compression bombs," in which some
compression algorithms are more or less susceptible too based on the
underlying design of the algorithm.

BER allows you to mix definite and indefinite length encodings - such as
creating CONSTRUCTED strings (which are indefinite length) which are made
up of multiple substrings of the same type (and which are definite length).
This gets especially hairy when you consider BITSTRINGs and the use of
unassigned bits - you can easily create a CONSTRUCTED BITSTRINGs with 3, 7,
3, 5 unused bits, and then have to worry even more about alignment and
correction. I know neither NSS nor CryptoAPI handle this case - even though
it's fully legal in BER.

While I can't speak for other browsers, I know that NSS would like to move
away from exposing BER in any API expect those directly related to S/MIME.
As I mentioned in my previous mail, the past 3 NSS CRITICAL severity bugs -
bugs that allowed for full remote code execution - were in the BER decoder.
These bugs existed for years. If you study the history of msasn1.dll, you
will see that Microsoft had a similar decade of hell of RCEs. You can find
similar stories in virtually every library that tries to write a compliant
BER parser.

What you end up with, then, as a result, is non-compliant BER parsers. And
now we're back to the central problem, which is that difference of
behaviour is meaningful and non-interoperable.

There is no legitimate reason to support BER. Chrome will not support BER
parsing - it is too open-ended, too security-risky, and there is zero
utility for a BER parser, in practice, outside of the CMS/SMIME space. If
you survey cryptographic libraries, you will find this. This is true for
decoding and encoding.

To that end, NSS and CryptoAPI are "special snowflakes", and they only have
the (limited, and in the case of NSS, unacceptable-for-security) BER
parsers they do to support CMS/SMIME, not because of SPKI/PKCS8.