W3C home > Mailing lists > Public > public-exi@w3.org > September 2008

Re: [EXI] Partial flush

From: Jochen Darley <joda@upb.de>
Date: Thu, 25 Sep 2008 22:07:10 +0200
Message-ID: <48DBEF6E.1010608@upb.de>
To: Yuri Delendik <yury_exi@yahoo.com>
CC: Daniel Peintner <daniel.peintner@gmail.com>, public-exi@w3.org

Hello Yuri & Daniel,

Yuri Delendik wrote:
> By buffer I meant 1 byte (8 bits). Now there is no way for EXI encoder send incomplete byte to the underline channel.
>  
> Let’s use XMPP example. XML stream data will be send in following manner without closing the underline channel: XML, {time delay}, XML, {time delay}, …, XML.
>  
> For example:
> <stream>
> {some time delay}
> <stanza1>…</stanza1>
> {some time delay}
> <stanza2>…</stanza2>
> <stanza1>…</stanza1>
> {some time delay}
> ….
> </stream>
>  
> And EXI encoder may hold bits after <stream> or </stanza1> and not send them to the channel, but other party is expecting that data as soon as possible -- those stanzas can be time critical material. 
>  
> ZLIB has partial flush functionality that aligns bits to byte border and sends compressed data to the channel, and makes it available to other party. But it does not terminate compression process.
>  
> Compression’s blockSize will not help is this case ether. 

This seem to be a valid point. Also I wonder why there can't be less
than blockSize AT,CH vales. The specification states every block has
to contain "...the minimum set of consecutive events that have _exactly
blockSize_ Attribute (AT) and Character (CH) values..." [3]. If I
understood that chapter correctly then EXI does not allow "flushing"
partial blocks, because such a block has less than "_exactly blockSize_"
AT and CH events.

Yuri used a good example with common use cases:
1) XML LOG streams
2) XML based news feeds
3) XMPP messages (over an uninterrupted HTTP connection)
....

With the current format limitations the data will be delayed by "hours"
and/or be incomplete, if I did not miss something.

I've been wondering if skipping could be supported by EXI. XML does not
support skipping, but does not prevent it. For example after adding an
attribute with the (byte) size of the element's
content the content can be skipped during reading.

So I thought about the dependencies of blocks:

What do I need to read a block in the middle of the stream?
1) some global dependencies: Schema, Compression Options
2) incremental String Table (depends on many/all previous blocks?)
3) ???

If I understood the spec correctly there is no block dependency for
classic compression (e.g. deflate), as each block is compressed separately.
As far as I can tell EXI does _prevent skipping_ as the string table can
not be reset or written to the stream.

Is it possible for EXI to have less dependencies between blocks ?
Compress a block (packet?) as much as possible, but keep blocks as
independent as possible would help to support skipping.

Thanks,
Jochen Darley

> [1] http://www.w3.org/TR/exi/#compression
> [2] http://www.w3.org/TR/exi/#key-blockSizeOption
[3] http://www.w3.org/TR/exi/#blocks
Received on Thursday, 25 September 2008 20:08:33 UTC

This archive was generated by hypermail 2.3.1 : Tuesday, 6 January 2015 19:52:43 UTC