RE: ISSUE 22 - Re: Incomplete blocks

Dear all,

We tried during our last call to close the issue 18 related to crypto operation, mentioning the streaming operation based on URI semantic [see issue description under http://www.w3.org/2012/webcrypto/track/issues/18] .
Some folks mentioned that the conversation about the incomplete blocks may be also related to it.

Could you confirm if 'incomplete block' relates to issue 18 ? or is an independent threat ?

Regards,
Virginie



From: Aymeric Vitte [mailto:vitteaymeric@gmail.com]
Sent: lundi 18 février 2013 11:58
To: Eric Rescorla
Cc: Ryan Sleevi; Richard Barnes; public-webcrypto-comments@w3.org
Subject: Re: ISSUE 22 - Re: Incomplete blocks


Le 17/02/2013 16:05, Eric Rescorla a écrit :
On Sun, Feb 17, 2013 at 2:09 AM, Aymeric Vitte <vitteaymeric@gmail.com<mailto:vitteaymeric@gmail.com>> wrote:
Yes, let's see what Ryan says but :

- ISSUE-22 still apply to hash (maybe a finish that does not really finish instead of a clone ?)

Hashes and dencryption aren't the same.

I know...




- I thought I understood that process could emit different progress as mentioned below, then maybe I can not know exactly when the last data have been consumed

Yes, i am saying that I think that that process should behave deterministically.

If different progress are emitted, for ctr it's easy to know when all data have been consumed but maybe not for other modes.




- case of encryption with padding (does it make sense to have progressive encryption in that case ?)

Yes.

Then ISSUE-22 is about encryption too.



-Ekr

Regards,
Le 16/02/2013 20:47, Eric Rescorla a écrit :
Aymeric,

If I understand the problem correctly, my view matches what I think Ryans is, namely
that the API should guarantee that as many bytes of data be consumed as possible
by the encryption and decryption process. Specifically:

- For stream and counter mode ciphers if X bytes are supplied, X bytes are output.
- For block ciphers, if X bytes are supplied X * floor(X/blocksize) are output.

Obviously, any AEAD mode cipher will need a finalize method of somesort.

-Ekr


On Sat, Feb 16, 2013 at 11:16 AM, Aymeric Vitte <vitteaymeric@gmail.com<mailto:vitteaymeric@gmail.com>> wrote:
Let's try again and let's try to simplify :

My encryption algorithm is processing 4 blocks  :

stream 1 : AABBCCDDEE --> final : aabbccddee
stream 2 : FFGGHHIIJJ --> final : ffgghhiijj

stream 1 + stream 2: AABBCCDDEEFFGGHHIIJJ --> final : AA_BB_CC_DD_EE_FF_GG_HH_II_JJ_

Now, progressive encryption :

"openssl"
stream 1 : AABBCCDDEE --> update : AA_BB_CC_DD_EE_
stream 2 : FFGGHHIIJJ --> update : FF_GG_HH_II_JJ_
(second result is the stream2 part of stream1+stream2 encrypted above

"cryptoJS"
stream 1 : AABBCCDDEE --> update : AA_BB_CC_DD_
stream 2 : FFGGHHIIJJ --> update : EE_FF_GG_HH_
So that's not the expected results, the workaround is :
"cryptoJS"
stream 1 : AABBCCDDEE --> update : AA_BB_CC_DD_--> clone and call final: AA_BB_CC_DD_EE
stream 2 : FFGGHHIIJJ --> clone.update : EE_FF_GG_HH_ --> clone2 and call clone.final, result is stream2 bytes of the result (last 5) : FF_GG_HH_II_JJ

But you mention : "Under the model, process always consumes all of the data given to it."

Then cryptoJS looks not correct. Now as far as I understand process could emit different progress for the same operation (AA_BB_CC_DD_, then EE_), then it's not clear how I can know when all data have been consumed.

But cryptoJS mentioned that in the case of padding:"Since CryptoJS might need to apply a padding, it can't encrypt any partial blocks until it knows whether it's the last block. Calling finalize() is how CryptoJS knows there are no more blocks coming, and it can then process the remaining partial block."

So in that case you call final and you close the progressive encryption that you can not recover unless using a clone like method.

I am not a fan of clone too, as you say it introduces security issues, for now that's the workaround I have used both for hash and cryptoJS encryption because I had no other choice.

Now, if "Under the model, process always consumes all of the data given to it.", and padding case of cryptoJS does not apply for progressive encryption and I can know from progress when all data have been consumed, then there is indeed no problem.

The issue here came from the fact that cryptoJS's update does not consume all data, and I just didn't know if it was "authorized" or not, but you say it's not.

Regards,


Le 16/02/2013 01:03, Ryan Sleevi a écrit :

I'm sorry, I've read this several times, and the related bug, and am
still having trouble what you're asking about or why you feel .clone()
is appropriate here.

.clone() is something especially dangerous for encryption, given that
for most systems, it will result in a catastrophic failure (eg: due to
IV reuse).

// Using pseudo-code here, not the actual API
var a = window.crypto.encrypt(..., {... { iv: 1 } })
a.process('abcd');  // Encrypts under IV 1, Increments IV to 2
var b = a.clone();  // b.iv == 2
a.process('efgh');  // Encrypts under IV 2, increments IV to 3
b.process('ijlk');  // Encrypts under **IV 2**, increments IV to **3**

In this case, a and b have no collided under IVs for the same key.
Very, very bad things happen.

Under the model, process always consumes all of the data given to it.
As best I can tell, this is your "OpenSSL" example. But it's not clear
at all based on your description, so it would be helpful if you could
try to simplify your example with the actual primitives needed.

On Fri, Feb 15, 2013 at 3:43 PM, Aymeric Vitte <vitteaymeric@gmail.com<mailto:vitteaymeric@gmail.com>> wrote:
Let's try... basically I am saying that ISSUE 22 is not only about hash but
encryption too and the conclusion should be that a clone method should be
added.

cryptoJS behaviour is a good example, as stated in issue 73, update does not
process all the blocks even if it could (ie no padding), you have to call
final to get all the blocks processed.

But other implementations like openssl do not behave the same, update does
return all the blocks processed.

Then for example, if you take a progressive encryption like tor protocol
with aes-ctr :

openssl : stream1  (509 bytes) --> update --> stream1 encrypted (result=509
bytes encrypted - 0 byte remaining)
cryptoJS : stream 1 (509 bytes) --> update -->  stream 1 encrypted
(result=496 bytes encrypted - 13 bytes remaining)

openssl : stream2  (509 bytes) --> update --> continue encryption with 0
remaining byte - result = 509 bytes (corresponds to the last 509 bytes of
stream1+stream2 encrypted - 0 byte remaining)
cryptoJS : stream 2 (509 bytes) --> update --> continue encryption with 13
remaining bytes - result = 512 bytes (corresponds to the last 512 bytes of
496 bytes of stream1+13 remaining bytes+ part of stream2 (499 bytes)
encrypted - 10 bytes remaining)

So, with the cryptoJS behaviour, only 496 bytes of the initial 509 bytes
would be encrypted and sent, then stream2 would contain the 13 last
encrypted bytes of stream1 + 499 encrypted bytes of stream2.

Of course, since each stream might not contain only pure streamed
information (like file, img, etc) but can contain instructions (like
encrypted(connect to mydomain.com<http://mydomain.com>)), you do not expect to receive these
instructions in different parts that you can not reconciliate, and you can
not wait for stream2 if you detect that stream1 encryption is not complete,
because stream2 might depend on stream1 action, therefore never come.

If you call final at each step, then you close the encryptor and just get
stream1 encrypted, then stream2 encrypted (not last 509 bytes of
stream1+stream2 encrypted), etc

The solution here is issue 74, ie clone.

I did not invent it, that's the way it's working with tor protocol, I have
some hard time understanding why the stream length chosen is not a multiple
of something that could be computed by an update without any potential
remaining bytes, or what is the official policy for update (should it return
whatever blocks it can process or not), but that's the way it is, and again
it's not something from myself.

Regards,



Le 15/02/2013 19:49, Ryan Sleevi a écrit :
On Fri, Feb 15, 2013 at 5:55 AM, Aymeric Vitte <vitteaymeric@gmail.com<mailto:vitteaymeric@gmail.com>>
wrote:
This reminds me that I should have sent an erratum of my erratum sent for
the encryption case related to Issue 22.

See http://code.google.com/p/crypto-js/issues/detail?id=73#c3 , issue
addressed to cryptoJS and finally accepted.

And see following issue (clone) :
http://code.google.com/p/crypto-js/issues/detail?id=74

This is a real life use case, current implementation of cryptoJS,
contrarly
to others, does not process all blocks when it can on "update", then you
have to call "final" which closes the encryptor (same as finish below).

I don't know who is right or wrong and if there is an official rule for
this, but it does not seem unlogical that cryptoJS "update" returns a
partial result (same as Ryan explained for process/progress results which
are let to the appreciation of the UA), even if other implementations do
return "final".

But then I can not achieve what I want to do, and I must use a clone
method
for this.

So, Issue 22 can be about encryption too, probably a clone method is
needed.
I'm sorry Aymeric, but having both read your reply and the bug, I'm
having trouble understanding what it is you're actually asking or
suggesting is a bug, nor what you're trying to do (or if it even makes
sense from a cryptographic security perspective).

Could you perhaps try restating?

--
jCore
Email :  avitte@jcore.fr<mailto:avitte@jcore.fr>
iAnonym : http://www.ianonym.com
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms
Web :    www.jcore.fr<http://www.jcore.fr>
Webble : www.webble.it<http://www.webble.it>
Extract Widget Mobile : www.extractwidget.com<http://www.extractwidget.com>
BlimpMe! : www.blimpme.com<http://www.blimpme.com>

--
jCore
Email :  avitte@jcore.fr<mailto:avitte@jcore.fr>
iAnonym : http://www.ianonym.com
node-Tor : https://www.github.com/Ayms/node-Tor
GitHub : https://www.github.com/Ayms
Web :    www.jcore.fr<http://www.jcore.fr>
Webble : www.webble.it<http://www.webble.it>
Extract Widget Mobile : www.extractwidget.com<http://www.extractwidget.com>
BlimpMe! : www.blimpme.com<http://www.blimpme.com>





--

jCore

Email :  avitte@jcore.fr<mailto:avitte@jcore.fr>

iAnonym : http://www.ianonym.com

node-Tor : https://www.github.com/Ayms/node-Tor

GitHub : https://www.github.com/Ayms

Web :    www.jcore.fr<http://www.jcore.fr>

Webble : www.webble.it<http://www.webble.it>

Extract Widget Mobile : www.extractwidget.com<http://www.extractwidget.com>

BlimpMe! : www.blimpme.com<http://www.blimpme.com>




--

jCore

Email :  avitte@jcore.fr<mailto:avitte@jcore.fr>

iAnonym : http://www.ianonym.com

node-Tor : https://www.github.com/Ayms/node-Tor

GitHub : https://www.github.com/Ayms

Web :    www.jcore.fr<http://www.jcore.fr>

Webble : www.webble.it<http://www.webble.it>

Extract Widget Mobile : www.extractwidget.com<http://www.extractwidget.com>

BlimpMe! : www.blimpme.com<http://www.blimpme.com>