Re: ISSUE 22 - Re: Incomplete blocks

Aymeric,

If I understand the problem correctly, my view matches what I think Ryans
is, namely
that the API should guarantee that as many bytes of data be consumed as
possible
by the encryption and decryption process. Specifically:

- For stream and counter mode ciphers if X bytes are supplied, X bytes are
output.
- For block ciphers, if X bytes are supplied X * floor(X/blocksize) are
output.

Obviously, any AEAD mode cipher will need a finalize method of somesort.

-Ekr


On Sat, Feb 16, 2013 at 11:16 AM, Aymeric Vitte <vitteaymeric@gmail.com>wrote:

> Let's try again and let's try to simplify :
>
> My encryption algorithm is processing 4 blocks  :
>
> stream 1 : AABBCCDDEE --> final : aabbccddee
> stream 2 : FFGGHHIIJJ --> final : ffgghhiijj
>
> stream 1 + stream 2: AABBCCDDEEFFGGHHIIJJ --> final :
> AA_BB_CC_DD_EE_FF_GG_HH_II_JJ_
>
> Now, progressive encryption :
>
> "openssl"
> stream 1 : AABBCCDDEE --> update : AA_BB_CC_DD_EE_
> stream 2 : FFGGHHIIJJ --> update : FF_GG_HH_II_JJ_
> (second result is the stream2 part of stream1+stream2 encrypted above
>
> "cryptoJS"
> stream 1 : AABBCCDDEE --> update : AA_BB_CC_DD_
> stream 2 : FFGGHHIIJJ --> update : EE_FF_GG_HH_
> So that's not the expected results, the workaround is :
> "cryptoJS"
> stream 1 : AABBCCDDEE --> update : AA_BB_CC_DD_--> clone and call final:
> AA_BB_CC_DD_EE
> stream 2 : FFGGHHIIJJ --> clone.update : EE_FF_GG_HH_ --> clone2 and call
> clone.final, result is stream2 bytes of the result (last 5) : FF_GG_HH_II_JJ
>
> But you mention : "Under the model, process always consumes all of the
> data given to it."
>
> Then cryptoJS looks not correct. Now as far as I understand process could
> emit different progress for the same operation (AA_BB_CC_DD_, then EE_),
> then it's not clear how I can know when all data have been consumed.
>
> But cryptoJS mentioned that in the case of padding:"Since CryptoJS might
> need to apply a padding, it can't encrypt any partial blocks until it knows
> whether it's the last block. Calling finalize() is how CryptoJS knows there
> are no more blocks coming, and it can then process the remaining partial
> block."
>
> So in that case you call final and you close the progressive encryption
> that you can not recover unless using a clone like method.
>
> I am not a fan of clone too, as you say it introduces security issues, for
> now that's the workaround I have used both for hash and cryptoJS encryption
> because I had no other choice.
>
> Now, if "Under the model, process always consumes all of the data given to
> it.", and padding case of cryptoJS does not apply for progressive
> encryption and I can know from progress when all data have been consumed,
> then there is indeed no problem.
>
> The issue here came from the fact that cryptoJS's update does not consume
> all data, and I just didn't know if it was "authorized" or not, but you say
> it's not.
>
> Regards,
>
>
> Le 16/02/2013 01:03, Ryan Sleevi a écrit :
>
>  I'm sorry, I've read this several times, and the related bug, and am
>> still having trouble what you're asking about or why you feel .clone()
>> is appropriate here.
>>
>> .clone() is something especially dangerous for encryption, given that
>> for most systems, it will result in a catastrophic failure (eg: due to
>> IV reuse).
>>
>> // Using pseudo-code here, not the actual API
>> var a = window.crypto.encrypt(..., {... { iv: 1 } })
>> a.process('abcd');  // Encrypts under IV 1, Increments IV to 2
>> var b = a.clone();  // b.iv == 2
>> a.process('efgh');  // Encrypts under IV 2, increments IV to 3
>> b.process('ijlk');  // Encrypts under **IV 2**, increments IV to **3**
>>
>> In this case, a and b have no collided under IVs for the same key.
>> Very, very bad things happen.
>>
>> Under the model, process always consumes all of the data given to it.
>> As best I can tell, this is your "OpenSSL" example. But it's not clear
>> at all based on your description, so it would be helpful if you could
>> try to simplify your example with the actual primitives needed.
>>
>> On Fri, Feb 15, 2013 at 3:43 PM, Aymeric Vitte <vitteaymeric@gmail.com>
>> wrote:
>>
>>> Let's try... basically I am saying that ISSUE 22 is not only about hash
>>> but
>>> encryption too and the conclusion should be that a clone method should be
>>> added.
>>>
>>> cryptoJS behaviour is a good example, as stated in issue 73, update does
>>> not
>>> process all the blocks even if it could (ie no padding), you have to call
>>> final to get all the blocks processed.
>>>
>>> But other implementations like openssl do not behave the same, update
>>> does
>>> return all the blocks processed.
>>>
>>> Then for example, if you take a progressive encryption like tor protocol
>>> with aes-ctr :
>>>
>>> openssl : stream1  (509 bytes) --> update --> stream1 encrypted
>>> (result=509
>>> bytes encrypted - 0 byte remaining)
>>> cryptoJS : stream 1 (509 bytes) --> update -->  stream 1 encrypted
>>> (result=496 bytes encrypted - 13 bytes remaining)
>>>
>>> openssl : stream2  (509 bytes) --> update --> continue encryption with 0
>>> remaining byte - result = 509 bytes (corresponds to the last 509 bytes of
>>> stream1+stream2 encrypted - 0 byte remaining)
>>> cryptoJS : stream 2 (509 bytes) --> update --> continue encryption with
>>> 13
>>> remaining bytes - result = 512 bytes (corresponds to the last 512 bytes
>>> of
>>> 496 bytes of stream1+13 remaining bytes+ part of stream2 (499 bytes)
>>> encrypted - 10 bytes remaining)
>>>
>>> So, with the cryptoJS behaviour, only 496 bytes of the initial 509 bytes
>>> would be encrypted and sent, then stream2 would contain the 13 last
>>> encrypted bytes of stream1 + 499 encrypted bytes of stream2.
>>>
>>> Of course, since each stream might not contain only pure streamed
>>> information (like file, img, etc) but can contain instructions (like
>>> encrypted(connect to mydomain.com)), you do not expect to receive these
>>> instructions in different parts that you can not reconciliate, and you
>>> can
>>> not wait for stream2 if you detect that stream1 encryption is not
>>> complete,
>>> because stream2 might depend on stream1 action, therefore never come.
>>>
>>> If you call final at each step, then you close the encryptor and just get
>>> stream1 encrypted, then stream2 encrypted (not last 509 bytes of
>>> stream1+stream2 encrypted), etc
>>>
>>> The solution here is issue 74, ie clone.
>>>
>>> I did not invent it, that's the way it's working with tor protocol, I
>>> have
>>> some hard time understanding why the stream length chosen is not a
>>> multiple
>>> of something that could be computed by an update without any potential
>>> remaining bytes, or what is the official policy for update (should it
>>> return
>>> whatever blocks it can process or not), but that's the way it is, and
>>> again
>>> it's not something from myself.
>>>
>>> Regards,
>>>
>>>
>>>
>>> Le 15/02/2013 19:49, Ryan Sleevi a écrit :
>>>
>>>  On Fri, Feb 15, 2013 at 5:55 AM, Aymeric Vitte <vitteaymeric@gmail.com>
>>>> wrote:
>>>>
>>>>> This reminds me that I should have sent an erratum of my erratum sent
>>>>> for
>>>>> the encryption case related to Issue 22.
>>>>>
>>>>> See http://code.google.com/p/**crypto-js/issues/detail?id=73#**c3<http://code.google.com/p/crypto-js/issues/detail?id=73#c3>, issue
>>>>> addressed to cryptoJS and finally accepted.
>>>>>
>>>>> And see following issue (clone) :
>>>>> http://code.google.com/p/**crypto-js/issues/detail?id=74<http://code.google.com/p/crypto-js/issues/detail?id=74>
>>>>>
>>>>> This is a real life use case, current implementation of cryptoJS,
>>>>> contrarly
>>>>> to others, does not process all blocks when it can on "update", then
>>>>> you
>>>>> have to call "final" which closes the encryptor (same as finish below).
>>>>>
>>>>> I don't know who is right or wrong and if there is an official rule for
>>>>> this, but it does not seem unlogical that cryptoJS "update" returns a
>>>>> partial result (same as Ryan explained for process/progress results
>>>>> which
>>>>> are let to the appreciation of the UA), even if other implementations
>>>>> do
>>>>> return "final".
>>>>>
>>>>> But then I can not achieve what I want to do, and I must use a clone
>>>>> method
>>>>> for this.
>>>>>
>>>>> So, Issue 22 can be about encryption too, probably a clone method is
>>>>> needed.
>>>>>
>>>> I'm sorry Aymeric, but having both read your reply and the bug, I'm
>>>> having trouble understanding what it is you're actually asking or
>>>> suggesting is a bug, nor what you're trying to do (or if it even makes
>>>> sense from a cryptographic security perspective).
>>>>
>>>> Could you perhaps try restating?
>>>>
>>>
>>> --
>>> jCore
>>> Email :  avitte@jcore.fr
>>> iAnonym : http://www.ianonym.com
>>> node-Tor : https://www.github.com/Ayms/**node-Tor<https://www.github.com/Ayms/node-Tor>
>>> GitHub : https://www.github.com/Ayms
>>> Web :    www.jcore.fr
>>> Webble : www.webble.it
>>> Extract Widget Mobile : www.extractwidget.com
>>> BlimpMe! : www.blimpme.com
>>>
>>>
> --
> jCore
> Email :  avitte@jcore.fr
> iAnonym : http://www.ianonym.com
> node-Tor : https://www.github.com/Ayms/**node-Tor<https://www.github.com/Ayms/node-Tor>
> GitHub : https://www.github.com/Ayms
> Web :    www.jcore.fr
> Webble : www.webble.it
> Extract Widget Mobile : www.extractwidget.com
> BlimpMe! : www.blimpme.com
>
>
>