Re: Streaming - [Re: CryptoOperation and its life cycle]

Please do not use HTML mail when replying to W3C lists. It messes up
the threading and archiving.

On Sun, Dec 16, 2012 at 10:36 AM, Aymeric Vitte <vitteaymeric@gmail.com> wrote:
> Trying to summarize here the different emails :
>
> Le 14/12/2012 19:52, Ryan Sleevi a écrit :
>
> Then what you want has nothing to do with "streaming", nor
> "progressive", nor "multi-part".
>
> Yes it has (or please re-explain your concept of streaming)

Streaming has to do with the use of HTML5 Streams (see the Streams
API) or the HTML5 Stream-like objects (see the File API, which defines
Files and Blobs)

>
>
> What you want has nothing to do with the behaviour/implementation of
> .process()
>
> See below
>
>
> What you want relates to ISSUE-22 (
> http://www.w3.org/2012/webcrypto/track/issues/22 ): Cloning.It's very clear
> you're cloning the digest context via the call to
> EVP_MD_CTX_copy. That is the same as a clone method.
>
>
> No, I (or others) am(are) not cloning the CryptoOperation, just the context
> (which is a kind of not so nice hook/hack), nothing to do with a clone
> method, which does not exist in js/TC39

Yes, is is. It is the exact same. The CryptoOperation *is* the context.

I have no clue where or why you mentioned TC39, this is about a method
being introduced onto the CryptoOperation to allow it to clone the
state of the underlying operation.

As has been repeatedly explained though, an underlying implementation
is not required to support multi-part operations, so it's entirely
possible that

x.process(stream1);
x2 = x.clone();
x1.finish();
x2.process(stream2);
x2.finish();

Results in just as much memory usage by the underlying implementation as

x.process(stream1);
x.finish();
x2.process(stream1);
x2.process(stream2);
x2.finish();

You're presuming there's an optimization in supporting cloning, but
that is not the case.

>
> Le 14/12/2012 19:54, Ryan Sleevi a écrit :
>
> I already explained to you how intermediate results are made available
> from methods that do support intermediate results - such as
> encryption.
>
> No, you just explained that progress can fire whenever the UA likes it, or I
> missed it, then please explain how we can get the intermediate results "1:1"

Again, there is no API to get the results 1:1.

You may .process(2400 bytes) and get an .onprogress(1600 bytes),
.onprogress(800 bytes) - depending on the user agent.

>
> Digesting does not support intermediate results. You MUST clone the
> digest state, because there is a destructive finalization step.
> Cloning is ISSUE-22.

You again ignored that digesting has no intermediate state.

>
>>
>> Maybe a solution could be to reuse the crypto operation after finish (and
>> then add something like a destroy method to CryptoOperation)
>
> This is not possible.
>
> The spec does not say what happens to the crypto operation after finish

Regardless of what the API says, for how you want to use it, it does
not do what you want. As has been repeatedly explained, hash
finalization is destructive of the state due to the addition of
padding.

Very early on the talk of object re-use was proposed - not for
purposes of extending processing (which, again, is cloning) - but as a
way for webapps to try to prematurely optimize. That feature was
removed from the Editor's Draft because it's demonstrable that web
apps should not have to worry about that, nor can they effectively
optimize. Exposing it to web apps significantly reduces the ability of
the UA to make informed decisions, which is why it was removed.

But again, that has no bearing on your request regarding incremental hashing.

>
>>
>> With simple ArrayBuffers, what's the problem with calculating separately
>> the digest over A and A|B ? ( | = concatenation). Is it just to optimize so
>> that we don't do the work for A twice ?
>
> Yes. That is the use case. Until you finalize a digest, you can clone
> the state and re-use it.
>
> For HMAC, for example, the use case was given that you can compute the
> IPAD/OPAD expansion of the key, and then clone that state, as a way to
> reduce the number of computations you have to do.
>
> The same would apply for storing the intermediate state of digest(A),
> before it had been finalized (by appending the padding and length).
> That way, the implementation does not have to recompute digest(A`)
> when computing digest(A|B).
>
> This is a valid use case, no question, but it's a potentially
> problematic one, hence why it has not *yet* been addressed in the
> Editor's Draft, and exists as ISSUE-22 -
> http://www.w3.org/2012/webcrypto/track/issues/22
>
> Issue 22 is talking about cloning the crypto operation, not the state,
> cloning the state could occur after finish and being reused, this seams
> easier than any cloning operation.

The CryptoOperation is the state.

Please re-read the specifications for the SHA* family of algorithms,
which are the currently provided digests. You will see that they have
an explicit finalization step that adds padding. This padding makes it
no longer possible to compute H(stream1 | stream2), because it's
really computing H(stream1 | padding | stream2)

Thus, ANY digest operation using these that has been finalized has
destructively altered the state. The only way to allow re-use would be
to require that a UA clone the internal state prior to a finish and
preserve it. That's an implementation pessimissm that I'm not willing
to make. clone() makes such an intent explicit, so it's at least a
reasonable solution, but it has its' own host of issues with relation
to the issues pointed out on ISSUE-22.

Again, you're talking about ISSUE-22 here.

Received on Sunday, 16 December 2012 19:24:03 UTC