Re: Low-Level API naming (was: Strawman proposal for the low-level API)

On Mon, Jul 16, 2012 at 9:59 AM, Lu HongQian Karen <karen.lu@gemalto.com> wrote:
> Hi Ryan,
>
>
>
> KL>> 4.  For algorithms, may be add start() to initiate the algorithm? The
> start initialized the algorithm. It also allows the caller to cancel the
> processData and restart again with the same context.
>
>
>
> RS> I'm not sure I understand how you see this working. Perhaps you could
> provide some pseudo-code that shows how it might be used?
>
>
>
> RS> Right now, the algorithm is implicitly started by virtue of creating -
> the caller can immediately call processData on it. If there is some state
> added between those two (NOT_STARTED -> STARTED), what do you see callers
> doing in the NOT_STATED case?
>
>
>
> The caller would do nothing in the non-started state, except start().
> Basically, we can have operations like the following:
>
>
>
> cryptOp.start();
>
> loop:
>
>    cryptOp.processData(msg);
>
> cryptOp.complete();
>
> cryptOp.start();
>
> cryptOp.processData(msg);
>
> cryptOp.complete();
>
>
>
> cryptOp.start();
>
> loop:
>
>    cryptOp.processData(message);
>
> cryptOp.start(); // there is a problem, restart
>
> … …
>
>
>
> The start() is a convenience feature. Without the start, when there is a
> problem, we could initiate another cryptostream. Do we need to worry about
> the old one, which processed part of the message? Even without a problem,
> maybe it is more convenient to be able to reuse an existing cryptostream?
>
>
>
> Many of us are familiar with init()/end() functions for algorithms or
> processes. Here start=init; complete=end.
>
>
>
> Thanks,
>
> Karen

Karen,

Thanks for explaining. From looking at your example, as I understand
it, you're proposing .start() explicitly so that cryptoOp can be
re-used for different purposes, is that correct? Is there any other
functional purpose of .start(), other than because other cryptographic
APIs have such methods (Init/Update/End) ?

It seems that re-using objects like you propose is not very common
within the JavaScript bindings, and when it exists, often creates a
lot more developer headache than it solves. For example, just looking
at the trouble people have had with XMLHttpRequest re-use (which
requires an explicit .abort() call) seems to suggest this is not
always clear. Also, by having more than one way to do it, it creates a
good deal of confusion for what is "The Right Way" - which can be
dependent on browsers' JS implementation, GC timing, etc.

Again, using the File API as a counter-point, each caller must create
a new FileReader() in order to satisfy some read request, and new
FileReaders should be created for new slices.

That said, as I mentioned on the discovery thread, I think .start()
may be useful for asynchronous discovery, since it offers a way for
the caller to register an onerror callback first, and then call
.start(), as opposed to the synchronous interface of .supports() or
throwing an exception during the call to window.crypto.encrypt(). But
I would think that's separate from the discussion of object re-use.

I'd also be concerned with object re-use-as-reinitialization since it
could lead to possible confusion about the state of an object. For
example, imagine an encryption operation using AES-CTR. The caller
specified the counter during window.crypto.encrypt(), and then the
algorithm maintains advancing the counter further through each
.processData() call (with M1, M2, M3, M4, etc).

If an application was confused and called .start() on that object
again, then it's possible that instead of IV1-M1, IV2-M2, IV3-M3,
IV4-M4, they could end up pairing IV1-M3, IV2-M4, etc. With the
"create a new object" idiom, such an above mistake would not be as
trivial to make (although it certainly may still exist)

It seems like your proposal could be the same as

var cryptOp = createMyCryptOp(...);
loop:
  cryptOp.processData(msg);
cryptOp.complete();
cryptOp = createMyCryptOp(...);
cryptOp.processData(msg);
cryptOp.complete();

cryptOp = createMyCryptOp(...);
loop:
  cryptOp.processData(message);
cryptOp.start();

Which is a common pattern for dealing with XMLHttpRequest, FileReader, etc.

What do you think?

>
>
>
> From: Ryan Sleevi [mailto:sleevi@google.com]
> Sent: Monday, July 02, 2012 8:39 PM
> To: Lu HongQian Karen
> Cc: public-webcrypto@w3.org; David Dahl
> Subject: Low-Level API naming (was: Strawman proposal for the low-level API)
>
>
>
> (Spinning up a new thread title for this part)
>
>
>
> On Mon, Jul 2, 2012 at 12:01 PM, Lu HongQian Karen <karen.lu@gemalto.com>
> wrote:
>
> Hi Ryan,
>
>
>
> Sorry for the late response. Thanks for putting things together. Here are my
> few cents.
>
>
>
> 1.  The low level API needs to include key handling functions, e.g. create
> keypair, create symm key, derive key, etc.
>
> Absolutely. As discussed on the call today, no strawman has been provided
> for this part of the API, beyond my rough sketch of the KeyQueryList API,
> which is hardly sufficient for all of the potential operations.
>
>
>
> 2.  The functions on the crypto object creates operators for respective
> crypto operations, called cryptostream’s. The concept is fine, but naming
> convention is confusing.
>
> a.  For example, I would expect encrypt() to encrypt something instead of
> creating an encryptor; May be simply change it to createEncryptor()
>
> Good point. I was originally going for an economy of characters, but a quick
> survey of the HTML5/W3C APIs suggests the pattern of either a Foo interface
> that is a constructor - eg (var enc = new Encryptor(...)) or using a create*
> method such as what you proposed, createEncryptor.
>
>
>
> The choice between object-with-Constructor and object-as-method was largely
> arbitrary, and primarily motivated by the fact that I didn't want to have a
> bunch of objects (Encryptor, Decryptor, Signer, Verifier) that were all
> identical in interface, and their only distinction being the semantic
> meaning given their names. However, this may be something to explore as a WG
> - which form is more preferable:
>
>
>
> a. var foo =
> window[.crypto?].createEncryptor()/createDecryptor/createSigner/createKeyQueryList/etc
>
> b. var foo = new Encryptor/Decryptor/Signer/Verifier/KeyDeriver/KeyQueryList
>
> c. var foo = new CryptoOperation("encrypt"/"decrypt"/"sign"/"verify")
>
> d. var foo = window.crypto.encrypt/decrypt/deriveKey/queryKey/etc
>
> b.  CryptoStream is also confusing because commonly used ciphers, such as
> AES, are block ciphers instead of stream ciphers. May be change it to
> CryptoOperator?
>
> Excellent point. I think the intent was to try to convey that it was a
> streaming operation, not a one-off, as you noted in comment a. I had
> originally considered CipherOp/CipherOperation, but then recanted as I
> thought that the API may, at some future point, involve the combination of
> multiple algorithms, and the singular cipher would be confusing.
>
>
>
> For naming, I wonder which is better:
>
> a. CryptoOperation
>
> b. CryptoOperator
>
> c. CryptoOp
>
> 3.  For signing, should have the option to separate hash from private key
> encryption.
>
> Right. I indicated how I imagined the semantics might work for this in my
> reply to Vijay's mail, under the old subject of "Strawman proposal for the
> low-level API"
>
>
>
> a.  Optionally have continue hashing giving previous result
>
> So one thought about this is to define how clonability works for the
> underlying CryptoStream. I suspect this will be addressed on an
> algorithm-by-algorithm basis, but the intent being to duplicate the internal
> state.
>
>
>
> The outstanding question is whether or not it is possible, using the various
> underlying cryptographic implementations that may be implementing this, and
> using the various algorithms, whether clonability can be well-defined. For
> hashing/signatures/MACs, I think it's fairly easy (duplicate the internal
> hash state). For encryption/decryption, it's also likely possible (duplicate
> the IV/CTR/block state).
>
>
>
> One of the challenges comes up with whether or not it's actually desirable
> to allow encryption/decryption contexts to be duplicated.
>
>
>
> The argument for cloning (implicit via equality/assignment, or explicit,
> such as via a .clone() method), is that you can use it to restart
> encryption/decryption state. For example, if decrypting a large file, say 50
> MB, from the FILE API to be sent via WebSockets, you may wish to only
> decrypt 1 MB at a time, ensuring that the 1 MB is sent before reading
> another 1 MB (using Blob.slice() to accomplish this). Because
> CryptoStream.result is accumulative (as currently pseudo-spec'd), this would
> eventually lead to 50MB in JS memory. If you could duplicate contexts
> without duplicating .result, you might be able to have the decryptor only
> hold 1MB at a time.
>
>
>
> The argument against it would be that it may encourage, promote, or make
> easy the ability to re-use internal state with an algorithm that should not
> (such as an IV in CTR mode), which would/could undermine the underlying
> cryptographic primitive and security guarantees. Notably, this is a bug in
> the user of this API, but it does raise a question of how "fool proof" this
> API should try to be.
>
>
>
> 4.  For algorithms, may be add start() to initiate the algorithm? The start
> initialized the algorithm. It also allows the caller to cancel the
> processData and restart again with the same context.
>
> I'm not sure I understand how you see this working. Perhaps you could
> provide some pseudo-code that shows how it might be used?
>
>
>
> Right now, the algorithm is implicitly started by virtue of creating - the
> caller can immediately call processData on it. If there is some state added
> between those two (NOT_STARTED -> STARTED), what do you see callers doing in
> the NOT_STATED case?
>
>
>
>
>
> Regards,
>
> Karen
>
>
>
>
>
> From: Ryan Sleevi [mailto:sleevi@google.com]
> Sent: Monday, June 18, 2012 12:53 PM
> To: public-webcrypto@w3.org
>
>
> Subject: Strawman proposal for the low-level API
>
>
>
> Hi all,
>
>
>
>    While I'm still in the process of learning WebIDL [1] and the W3C Manual
> of Style [2], I wanted to take a quick shot at drafting a strawman low-level
> API for discussion. We've discussed quite a bit about key management [3] and
> key discovery [4], not to mention early discussions about algorithm
> discovery [5], but I think it might be good to move to the point where we
> can talk a bit about how these might be all put together.
>
>
>
> First, a bit of the IDL definition, to set the stage. This is also using
> using ArrayBuffer from TypedArray [6], which I'm not sure if it's altogether
> appropriate, but it's been incorporated by reference into FileAPI [7], so it
> seems alright to use here.
>
>
>
> [interface]
>
> interface CryptoStream : EventTarget {
>
>   void processData(ArrayBuffer buffer);
>
>   void processData(DOMString data);
>
>   void complete();
>
>
>
>   readonly attribute (DOMString or ArrayBuffer)? result;
>
>
>
>   attribute [TreatNonCallableAsNull] Function? onerror;
>
>   attribute [TreatNonCallableAsNull] Function? onprogress;
>
>   attribute [TreatNonCallableAsNull] Function? oncomplete;
>
> };
>
>
>
> dictionary AlgorithmParams {
>
> };
>
>
>
> dictionary Algorithm {
>
>   DOMString name;
>
>   AlgorithmParams? params;
>
> };
>
>
>
> [NoInterfaceObject]
>
> interface Crypto {
>
>   CryptoStream encrypt(Algorithm algorithm, Key key);
>
>   CryptoStream decrypt(Algorithm algorithm, Key key);
>
>
>
>   // Also handles MACs
>
>   CryptoStream sign(Algorithm algorithm, Key key);
>
>   CryptoStream verify(Algorithm algorithm, Key key, ArrayBuffer signature);
>
>
>
>   CryptoStream digest(Algorithm algorithm);
>
>
>
>   // This interface TBD. See discussion below.
>
>   bool supports(Algorithm algorithm, optional Key key);
>
>
>
>   // Interfaces for key derivation/generation TBD.
>
> };
>
>
>
>
>
> As you can see, CryptoStream is used for all of the actual crypto
> operations. That's because, in looking at the operations, I think all of
> them will work on a series of calls to provide input, and the result of
> which is either: error, some data output, or operation complete.
>
>
>
> The real challenge, I think, lies in the AlgorithmParams structure, which is
> where all of the algorithm-specific magic happens. My belief is that we
> can/should be able to define this API independent of any specific
> AlgorithmParams - that is, we can define the generic state machine, error
> handling, discovery. Then, as a supplemental work (still within the scope of
> the primary goal), we define and enumerate how exactly specific algorithms
> are implemented within this state machine.
>
>
>
> To show how different AlgorithmParams might be implemented, here's some
> varies definitions:
>
>
>
> // For the 'RSA-PSS' algorithm.
>
> dictionary RsaPssParams : AlgorithmParams {
>
>   // The hashing function to apply to the message (eg: SHA1).
>
>   AlgorithmParams hash;
>
>   // The mask generation function (eg: MGF1-SHA1)
>
>    AlgorithmParams mgf;
>
>   // The desired length of the random salt.
>
>   unsigned long saltLength;
>
> };
>
>
>
> // For the 'RSA-OAEP' algorithm.
>
> dictionary RsaOaepParams : AlgorithmParams {
>
>   // The hash function to apply to the message (eg: SHA1).
>
>    AlgorithmParams hash;
>
>   // The mask generation function (eg: MGF1-SHA1).
>
>    AlgorithmParams mgf;
>
>   // The optional label/application data to associate with the signature.
>
>   DOMString? label = null;
>
> };
>
>
>
> // For the 'AES-GCM' algorithm.
>
> dictionary AesGcmParams : AlgorithmParams {
>
>   ArrayBufferView? iv;
>
>   ArrayBufferView? additional;
>
>   unsigned long tagLength;
>
> };
>
>
>
> // For the 'AES-CCM' algorithm.
>
> dictionary AesCcmParams : AlgorithmParams {
>
>   ArrayBufferView? nonce;
>
>   ArrayBufferView? additional;
>
>   unsigned long macLength;
>
> };
>
>
>
> // For the 'HMAC' algorithm.
>
> dictionary HmacParams : AlgorithmParams {
>
>   // The hash function to use (eg: SHA1).
>
>   AlgorithmParams hash;
>
> };
>
>
>
>
>
> The API behaviour is this:
>
> - If encrypt/decrypt/sign/verify/digest is called with an unsupported
> algorithm, throw InvalidAlgorithmError.
>
> - If " is called with an invalid key, throw InvalidKeyError.
>
> - If " is called with an invalid key/algorithm combination, throw
> UnsupportedAlgorithmError.
>
> - Otherwise, return a CryptoStream.
>
>
>
> For encrypt/decrypt
>
> - The caller calls processData() as data is available.
>
> - If the data can be en/decrypted, it will raise an onprogress event (event
> type TBD).
>
>   - If new (plaintext, ciphertext) data is available, .result will be
> updated. [This is similar to the FileStream API behaviour]
>
> - If the data cannot be en/decrypted, raise the onerror with an appropriate
> error
>
> - The caller calls .complete() once all data has been processed.
>
>   - If the final block validates (eg: no padding errors), call onprocess
> then oncomplete.
>
>   - If the final block does not validate, call onerror with an appropriate
> error.
>
>
>
> For authenticated encryption modes, for example, the .result may not contain
> any data until .complete has been called (with the result data).
>
>
>
> For sign/verify, it behaves similarly.
>
> - The caller calls processData() as data is available.
>
> - [No onprogress is called/needs to be called?]
>
> - The caller calls .complete() once all data has been processed
>
> - For sign, once .complete() is called, the signature is generated, and
> either onprogress+oncomplete or onerror is called. If successful, the
> resultant signature is in .result.
>
> - For verify, once .complete() is called, the signature is compared, and
> either onprogress+oncomplete or onerror is called. If the signatures
> successfully matched, .result will contain the input signature (eg: the
> constant-time comparison happens within the library). If the signatures
> don't match, .result will be null and the error handler will have been
> called.
>
>
>
> Finally, for digesting, it behaves like .sign/.verify in that no data is
> available until .complete() is called, and once .compete() is called, the
> resultant digest is in .result.
>
>
>
> What I haven't fully worked out is how key derivation/agreement will work -
> particularly if the result of some result of key agreement results in
> multiple keys (eg: how SSL/TLS key derivation works in PKCS#11). This is
> somewhat dependent on how we treat keys.
>
>
>
> Note that I left the Key type unspecified. It's not clear if this will be
> something like (Key or DOMString), indicating some either/or of handle / id,
> if it might be a dictionary type (with different naming specifiers, such as
> 'id' or 'uuid'), or if it will be a concrete type obtained via some other
> call (eg: .queryKeys()). I think that will be borne out over the next week
> or two as we continue to discuss key management/lifecycle.
>
>
>
> For a pseudo-code example:
>
>
>
> var stream = window.crypto.sign({ name: 'RSA-PSS', params: { hash: { name:
> 'sha1' }, mgf: { name: 'mgf-sha1' }, saltLength: 32 }}, key);
>
> stream.oncomplete = function(evt) { window.alert('The signature is ' +
> e.target.result); };
>
> stream.onerror = function(evt) { window.alert('Signing caused an error: ' +
> e.error); };
>
>
>
> var filereader = FileReader();
>
> reader.onload = function(evt) { stream.processData(evt.target.result);
> stream.complete(); }
>
> filereader.readAsArrayBuffer(someFile);
>
>
>
>
>
> The FileAPI is probably not the best example of why the iterative API
> (.processData() + .complete()) is used, since FileReader has the
> FileReader.result containing all of the processed data, but it's similar
> than demonstrating a streaming operation that may be using WebSockets [8] or
> PeerConnection [9].
>
>
>
> Note that I think during the process of algorithm specification, we can
> probably get away with also defining well-known shorthand. eg:
> 'RSA-PSS-SHA256' would mean that the hash is SHA-256, the mgf is
> MGF1-SHA256, and only the saltLength needs to be specified (or should it be
> implied?)
>
>
>
> Anyways, hopefully this straw-man is able to spark some discussion, and
> hopefully if it's not fatally flawed, I'll be able to finish adopting it to
> the W3C template for proper and ongoing discussions.
>
>
>
> Cheers,
>
> Ryan
>
>
>
> References:
>
> [1] http://www.w3.org/TR/WebIDL
>
> [2] http://www.w3.org/2001/06/manual/
>
> [3] http://lists.w3.org/Archives/Public/public-webcrypto/2012Jun/0050.html
>
> [4] http://lists.w3.org/Archives/Public/public-webcrypto/2012Jun/0007.html
>
> [5] http://lists.w3.org/Archives/Public/public-webcrypto/2012May/0070.html
>
> [6] http://www.khronos.org/registry/typedarray/specs/latest/
>
> [7] http://www.w3.org/TR/FileAPI/
>
> [8] http://www.w3.org/TR/2009/WD-websockets-20091222/
>
> [9] http://dev.w3.org/2011/webrtc/editor/webrtc.html
>
>

Received on Monday, 16 July 2012 18:35:22 UTC