Re: Is there a pure JS implementation of the WOFF2 Brotli codec?

Ah, those are good links, thanks!

- Mike

On Wed, Jul 15, 2015 at 10:07 AM, Jonathan Kew <jfkthame@gmail.com> wrote:

> On 15/7/15 17:46, Mike Kamermans wrote:
>
>> Yeah,that's the conclusion we had to draw too - debugging the running
>> process revealed that the emscripten'd library was allocating three
>> massive array buffers (~313MB each), just by loading the library, so
>> that's either a nasty emscripten bug, or could be a deferred allocation
>> pattern that works in C++ but turns into instant allocation in JS.
>>
>
> Note that Brotli makes use of a very large static array, of which there
> are separate copies in the encoder and decoder sources; see
>
>   https://github.com/google/brotli/blob/master/dec/dictionary.h
>   https://github.com/google/brotli/blob/master/enc/dictionary.h
>
> If the emscripten'd library is eagerly loading this into an array, and
> doing so separately for the encoder and decoder, that could easily account
> for at least a couple of massive array allocations. (Why three, though? No
> idea offhand.)
>
> JK
>
>
>> That said, a pure JS implementation would be great as "reality check"
>> too, so that for raw speed a (fixed) emscripten'd library is great, but
>> for sanity, a secondary implementation is available to hold emscripten
>> accountable to.
>>
>> - Mike
>>
>> On Tue, Jul 14, 2015 at 7:37 PM, Robert O'Callahan <robert@ocallahan.org
>> <mailto:robert@ocallahan.org>> wrote:
>>
>>     On Wed, Jul 15, 2015 at 11:25 AM, Mike Kamermans <nihongo@gmail.com
>>     <mailto:nihongo@gmail.com>> wrote:
>>
>>         On the off-chance that anyone knows, fontkit
>>         (https://github.com/devongovett/fontkit) is currently using an
>>         emscripten'ed version of the Brotli codec for handling WOFF2
>>         decoding, which turns what is a 34MB footprint process without
>>         WOFF2 decoding into a ~1000MB footprint process with... making
>>         it rather unusable for dealing with WOFF2 files in any
>>         meaningful production setting.
>>
>>
>>     That overhead sounds way too high. I guess something is wrong with
>>     emscripten or some other part of the workflow.
>>
>>     Rob
>>     --
>>     lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe
>>     uresyf toD
>>     selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
>>     rdsme,aoreseoouoto
>>     o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or
>>     rnea lurpr
>>     .a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt
>>     sstvr  esn
>>
>>
>>
>