RE: Low-Level API naming (was: Strawman proposal for the low-level API)

Hi Ryan,

KL>> 4.  For algorithms, may be add start() to initiate the algorithm? The start initialized the algorithm. It also allows the caller to cancel the processData and restart again with the same context.

RS> I'm not sure I understand how you see this working. Perhaps you could provide some pseudo-code that shows how it might be used?

RS> Right now, the algorithm is implicitly started by virtue of creating - the caller can immediately call processData on it. If there is some state added between those two (NOT_STARTED -> STARTED), what do you see callers doing in the NOT_STATED case?

The caller would do nothing in the non-started state, except start(). Basically, we can have operations like the following:

cryptOp.start();
loop:
   cryptOp.processData(msg);
cryptOp.complete();
cryptOp.start();
cryptOp.processData(msg);
cryptOp.complete();

cryptOp.start();
loop:
   cryptOp.processData(message);
cryptOp.start(); // there is a problem, restart
... ...

The start() is a convenience feature. Without the start, when there is a problem, we could initiate another cryptostream. Do we need to worry about the old one, which processed part of the message? Even without a problem, maybe it is more convenient to be able to reuse an existing cryptostream?

Many of us are familiar with init()/end() functions for algorithms or processes. Here start=init; complete=end.

Thanks,
Karen

From: Ryan Sleevi [mailto:sleevi@google.com]
Sent: Monday, July 02, 2012 8:39 PM
To: Lu HongQian Karen
Cc: public-webcrypto@w3.org; David Dahl
Subject: Low-Level API naming (was: Strawman proposal for the low-level API)

(Spinning up a new thread title for this part)

On Mon, Jul 2, 2012 at 12:01 PM, Lu HongQian Karen <karen.lu@gemalto.com<mailto:karen.lu@gemalto.com>> wrote:
Hi Ryan,

Sorry for the late response. Thanks for putting things together. Here are my few cents.


1.  The low level API needs to include key handling functions, e.g. create keypair, create symm key, derive key, etc.
Absolutely. As discussed on the call today, no strawman has been provided for this part of the API, beyond my rough sketch of the KeyQueryList API, which is hardly sufficient for all of the potential operations.


2.  The functions on the crypto object creates operators for respective crypto operations, called cryptostream's. The concept is fine, but naming convention is confusing.

a.  For example, I would expect encrypt() to encrypt something instead of creating an encryptor; May be simply change it to createEncryptor()
Good point. I was originally going for an economy of characters, but a quick survey of the HTML5/W3C APIs suggests the pattern of either a Foo interface that is a constructor - eg (var enc = new Encryptor(...)) or using a create* method such as what you proposed, createEncryptor.

The choice between object-with-Constructor and object-as-method was largely arbitrary, and primarily motivated by the fact that I didn't want to have a bunch of objects (Encryptor, Decryptor, Signer, Verifier) that were all identical in interface, and their only distinction being the semantic meaning given their names. However, this may be something to explore as a WG - which form is more preferable:

a. var foo = window[.crypto?].createEncryptor()/createDecryptor/createSigner/createKeyQueryList/etc
b. var foo = new Encryptor/Decryptor/Signer/Verifier/KeyDeriver/KeyQueryList
c. var foo = new CryptoOperation("encrypt"/"decrypt"/"sign"/"verify")
d. var foo = window.crypto.encrypt/decrypt/deriveKey/queryKey/etc

b.  CryptoStream is also confusing because commonly used ciphers, such as AES, are block ciphers instead of stream ciphers. May be change it to CryptoOperator?
Excellent point. I think the intent was to try to convey that it was a streaming operation, not a one-off, as you noted in comment a. I had originally considered CipherOp/CipherOperation, but then recanted as I thought that the API may, at some future point, involve the combination of multiple algorithms, and the singular cipher would be confusing.

For naming, I wonder which is better:
a. CryptoOperation
b. CryptoOperator
c. CryptoOp

3.  For signing, should have the option to separate hash from private key encryption.
Right. I indicated how I imagined the semantics might work for this in my reply to Vijay's mail, under the old subject of "Strawman proposal for the low-level API"


a.  Optionally have continue hashing giving previous result
So one thought about this is to define how clonability works for the underlying CryptoStream. I suspect this will be addressed on an algorithm-by-algorithm basis, but the intent being to duplicate the internal state.

The outstanding question is whether or not it is possible, using the various underlying cryptographic implementations that may be implementing this, and using the various algorithms, whether clonability can be well-defined. For hashing/signatures/MACs, I think it's fairly easy (duplicate the internal hash state). For encryption/decryption, it's also likely possible (duplicate the IV/CTR/block state).

One of the challenges comes up with whether or not it's actually desirable to allow encryption/decryption contexts to be duplicated.

The argument for cloning (implicit via equality/assignment, or explicit, such as via a .clone() method), is that you can use it to restart encryption/decryption state. For example, if decrypting a large file, say 50 MB, from the FILE API to be sent via WebSockets, you may wish to only decrypt 1 MB at a time, ensuring that the 1 MB is sent before reading another 1 MB (using Blob.slice() to accomplish this). Because CryptoStream.result is accumulative (as currently pseudo-spec'd), this would eventually lead to 50MB in JS memory. If you could duplicate contexts without duplicating .result, you might be able to have the decryptor only hold 1MB at a time.

The argument against it would be that it may encourage, promote, or make easy the ability to re-use internal state with an algorithm that should not (such as an IV in CTR mode), which would/could undermine the underlying cryptographic primitive and security guarantees. Notably, this is a bug in the user of this API, but it does raise a question of how "fool proof" this API should try to be.


4.  For algorithms, may be add start() to initiate the algorithm? The start initialized the algorithm. It also allows the caller to cancel the processData and restart again with the same context.
I'm not sure I understand how you see this working. Perhaps you could provide some pseudo-code that shows how it might be used?

Right now, the algorithm is implicitly started by virtue of creating - the caller can immediately call processData on it. If there is some state added between those two (NOT_STARTED -> STARTED), what do you see callers doing in the NOT_STATED case?


Regards,
Karen


From: Ryan Sleevi [mailto:sleevi@google.com<mailto:sleevi@google.com>]
Sent: Monday, June 18, 2012 12:53 PM
To: public-webcrypto@w3.org<mailto:public-webcrypto@w3.org>

Subject: Strawman proposal for the low-level API

Hi all,

   While I'm still in the process of learning WebIDL [1] and the W3C Manual of Style [2], I wanted to take a quick shot at drafting a strawman low-level API for discussion. We've discussed quite a bit about key management [3] and key discovery [4], not to mention early discussions about algorithm discovery [5], but I think it might be good to move to the point where we can talk a bit about how these might be all put together.

First, a bit of the IDL definition, to set the stage. This is also using using ArrayBuffer from TypedArray [6], which I'm not sure if it's altogether appropriate, but it's been incorporated by reference into FileAPI [7], so it seems alright to use here.

[interface]
interface CryptoStream : EventTarget {
  void processData(ArrayBuffer buffer);
  void processData(DOMString data);
  void complete();

  readonly attribute (DOMString or ArrayBuffer)? result;

  attribute [TreatNonCallableAsNull] Function? onerror;
  attribute [TreatNonCallableAsNull] Function? onprogress;
  attribute [TreatNonCallableAsNull] Function? oncomplete;
};

dictionary AlgorithmParams {
};

dictionary Algorithm {
  DOMString name;
  AlgorithmParams? params;
};

[NoInterfaceObject]
interface Crypto {
  CryptoStream encrypt(Algorithm algorithm, Key key);
  CryptoStream decrypt(Algorithm algorithm, Key key);

  // Also handles MACs
  CryptoStream sign(Algorithm algorithm, Key key);
  CryptoStream verify(Algorithm algorithm, Key key, ArrayBuffer signature);

  CryptoStream digest(Algorithm algorithm);

  // This interface TBD. See discussion below.
  bool supports(Algorithm algorithm, optional Key key);

  // Interfaces for key derivation/generation TBD.
};


As you can see, CryptoStream is used for all of the actual crypto operations. That's because, in looking at the operations, I think all of them will work on a series of calls to provide input, and the result of which is either: error, some data output, or operation complete.

The real challenge, I think, lies in the AlgorithmParams structure, which is where all of the algorithm-specific magic happens. My belief is that we can/should be able to define this API independent of any specific AlgorithmParams - that is, we can define the generic state machine, error handling, discovery. Then, as a supplemental work (still within the scope of the primary goal), we define and enumerate how exactly specific algorithms are implemented within this state machine.

To show how different AlgorithmParams might be implemented, here's some varies definitions:

// For the 'RSA-PSS' algorithm.
dictionary RsaPssParams : AlgorithmParams {
  // The hashing function to apply to the message (eg: SHA1).
  AlgorithmParams hash;
  // The mask generation function (eg: MGF1-SHA1)
   AlgorithmParams mgf;
  // The desired length of the random salt.
  unsigned long saltLength;
};

// For the 'RSA-OAEP' algorithm.
dictionary RsaOaepParams : AlgorithmParams {
  // The hash function to apply to the message (eg: SHA1).
   AlgorithmParams hash;
  // The mask generation function (eg: MGF1-SHA1).
   AlgorithmParams mgf;
  // The optional label/application data to associate with the signature.
  DOMString? label = null;
};

// For the 'AES-GCM' algorithm.
dictionary AesGcmParams : AlgorithmParams {
  ArrayBufferView? iv;
  ArrayBufferView? additional;
  unsigned long tagLength;
};

// For the 'AES-CCM' algorithm.
dictionary AesCcmParams : AlgorithmParams {
  ArrayBufferView? nonce;
  ArrayBufferView? additional;
  unsigned long macLength;
};

// For the 'HMAC' algorithm.
dictionary HmacParams : AlgorithmParams {
  // The hash function to use (eg: SHA1).
  AlgorithmParams hash;
};


The API behaviour is this:
- If encrypt/decrypt/sign/verify/digest is called with an unsupported algorithm, throw InvalidAlgorithmError.
- If " is called with an invalid key, throw InvalidKeyError.
- If " is called with an invalid key/algorithm combination, throw UnsupportedAlgorithmError.
- Otherwise, return a CryptoStream.

For encrypt/decrypt
- The caller calls processData() as data is available.
- If the data can be en/decrypted, it will raise an onprogress event (event type TBD).
  - If new (plaintext, ciphertext) data is available, .result will be updated. [This is similar to the FileStream API behaviour]
- If the data cannot be en/decrypted, raise the onerror with an appropriate error
- The caller calls .complete() once all data has been processed.
  - If the final block validates (eg: no padding errors), call onprocess then oncomplete.
  - If the final block does not validate, call onerror with an appropriate error.

For authenticated encryption modes, for example, the .result may not contain any data until .complete has been called (with the result data).

For sign/verify, it behaves similarly.
- The caller calls processData() as data is available.
- [No onprogress is called/needs to be called?]
- The caller calls .complete() once all data has been processed
- For sign, once .complete() is called, the signature is generated, and either onprogress+oncomplete or onerror is called. If successful, the resultant signature is in .result.
- For verify, once .complete() is called, the signature is compared, and either onprogress+oncomplete or onerror is called. If the signatures successfully matched, .result will contain the input signature (eg: the constant-time comparison happens within the library). If the signatures don't match, .result will be null and the error handler will have been called.

Finally, for digesting, it behaves like .sign/.verify in that no data is available until .complete() is called, and once .compete() is called, the resultant digest is in .result.

What I haven't fully worked out is how key derivation/agreement will work - particularly if the result of some result of key agreement results in multiple keys (eg: how SSL/TLS key derivation works in PKCS#11). This is somewhat dependent on how we treat keys.

Note that I left the Key type unspecified. It's not clear if this will be something like (Key or DOMString), indicating some either/or of handle / id, if it might be a dictionary type (with different naming specifiers, such as 'id' or 'uuid'), or if it will be a concrete type obtained via some other call (eg: .queryKeys()). I think that will be borne out over the next week or two as we continue to discuss key management/lifecycle.

For a pseudo-code example:

var stream = window.crypto.sign({ name: 'RSA-PSS', params: { hash: { name: 'sha1' }, mgf: { name: 'mgf-sha1' }, saltLength: 32 }}, key);
stream.oncomplete = function(evt) { window.alert('The signature is ' + e.target.result); };
stream.onerror = function(evt) { window.alert('Signing caused an error: ' + e.error); };

var filereader = FileReader();
reader.onload = function(evt) { stream.processData(evt.target.result); stream.complete(); }
filereader.readAsArrayBuffer(someFile);


The FileAPI is probably not the best example of why the iterative API (.processData() + .complete()) is used, since FileReader has the FileReader.result containing all of the processed data, but it's similar than demonstrating a streaming operation that may be using WebSockets [8] or PeerConnection [9].

Note that I think during the process of algorithm specification, we can probably get away with also defining well-known shorthand. eg: 'RSA-PSS-SHA256' would mean that the hash is SHA-256, the mgf is MGF1-SHA256, and only the saltLength needs to be specified (or should it be implied?)

Anyways, hopefully this straw-man is able to spark some discussion, and hopefully if it's not fatally flawed, I'll be able to finish adopting it to the W3C template for proper and ongoing discussions.

Cheers,
Ryan

References:
[1] http://www.w3.org/TR/WebIDL
[2] http://www.w3.org/2001/06/manual/
[3] http://lists.w3.org/Archives/Public/public-webcrypto/2012Jun/0050.html
[4] http://lists.w3.org/Archives/Public/public-webcrypto/2012Jun/0007.html
[5] http://lists.w3.org/Archives/Public/public-webcrypto/2012May/0070.html
[6] http://www.khronos.org/registry/typedarray/specs/latest/
[7] http://www.w3.org/TR/FileAPI/
[8] http://www.w3.org/TR/2009/WD-websockets-20091222/
[9] http://dev.w3.org/2011/webrtc/editor/webrtc.html