RE: Proposal for including EXI in XMPP

Hello Rumen

Thanks for your excellent input. I’ll respond to your comments one at a time:

> 1. For very constrained devices it might be worth considering making the EXI options negotiation using EXI as well. This can be done by using the most restrictive set of options during the negotiations e.g. bit-packed, valuePartitionCapacity = 0 etc. I still think the Youenn's suggestion for reusing the EXI Options Schema document makes a lot of sense. EXI/XMPP specific options such as sessionWideBuffers can be communicated in a separate stanza. For example, by redefining the way the options are communicated you can exclude useful EXI extensions that are to be defined in the future - such as the upcoming EXI Profile specification (http://www.w3.org/TR/exi-profile/) that I believe is very relevant for this draft.

I’ve not chosen to use the EXI Options schema, mainly because it does not identify schemas in an way that is appropriate for this case: The options only defines a schemaId option for specifying schema. This is not sufficient since first, schemaId is not part of the lexical syntax of XML (namespace is). Furthermore, it does not allow for a plurality of schemas, nor does it allow for different versions of the same schema (i.e. devices from same vendor but with different versions).

> 2. The dynamic exchange of schema definitions is very interesting feature. I think the requirement to use plain XML Schema files to communicate the schemas (following from the MD5 Hash identification) is very limiting. Clients can choose to upload the XML Schema files using EXI as well. XML Schema files are just XML documents that can be encoded in EXI as well (prefixes preserved option set). My experience in using EXI encoded XML Schema files for EXI processing shows that XSD files are heavily compressed and faster to generate the required EXI grammars

You’re absolutely correct. I’ve added a section (§2.6) describing how to upload EXI-compressed schema files. I’ve not chosen to use an OpenExi document format explicitly, since I do not know at this point to which extent that format is universally accepted. (Actually I cannot find a universally accepted EXI document format, or MIME code for that matter.) So, for the time being, I’ve only written ExiBody and ExiDocument as options, not knowing better.

Could anybody please advise about desired/accepted file formats?

> One more argument in favour of allowing for different ways to share the schema definitions is to use EXI specific formats such as the EXI Schema Definition (ESD) used by OpenEXI. That being said there should be a more reliable way to specify the schema version in use than MD5 Hash approach.

MD5 Hash is used in conjunction with target namespace of the schema. So, the identifier is the pair (namespace, md5 hash). It is assumed to be exceptionally difficult to create an update to a schema that would generate the same hash, so the hash is a simple way to allow the server to use different schema files for different clients, even though they have the same namespace (or schema id for that matter).

Sincerely,
Peter Waher



From: Rumen Kyusakov [mailto:kjussakov@gmail.com]
Sent: den 18 mars 2013 13:16
To: Peter Waher
Cc: Takuki Kamiya; FABLET Youenn (Youenn.Fablet@crf.canon.fr); Joachim Lindborg; John Schneider; mact-usa@att.net; Peter Saint-Andre; public-exi@w3.org; Stephen Williams <sdw@lig.net> (sdw@lig.net); XMPP Standards; Yusuke DOI
Subject: Re: Proposal for including EXI in XMPP

...
(incidentally send the previous email unfinished)

One more argument in favour of allowing for different ways to share the schema definitions is to use EXI specific formats such as the EXI Schema Definition (ESD) used by OpenEXI. That being said there should be a more reliable way to specify the schema version in use than MD5 Hash approach.

Best regards,
Rumen




On Mon, Mar 18, 2013 at 5:09 PM, Rumen Kyusakov <kjussakov@gmail.com<mailto:kjussakov@gmail.com>> wrote:
Hello Peter, all,

Thank you for putting together this draft!
Please consider few points that pop-up while I was reading it:
1. For very constrained devices it might be worth considering making the EXI options negotiation using EXI as well. This can be done by using the most restrictive set of options during the negotiations e.g. bit-packed, valuePartitionCapacity = 0 etc. I still think the Youenn's suggestion for reusing the EXI Options Schema document makes a lot of sense. EXI/XMPP specific options such as sessionWideBuffers can be communicated in a separate stanza. For example, by redefining the way the options are communicated you can exclude useful EXI extensions that are to be defined in the future - such as the upcoming EXI Profile specification (http://www.w3.org/TR/exi-profile/) that I believe is very relevant for this draft.

2. The dynamic exchange of schema definitions is very interesting feature. I think the requirement to use plain XML Schema files to communicate the schemas (following from the MD5 Hash identification) is very limiting. Clients can choose to upload the XML Schema files using EXI as well. XML Schema files are just XML documents that can be encoded in EXI as well (prefixes preserved option set). My experience in using EXI encoded XML Schema files for EXI processing shows that XSD files are heavily compressed and faster to generate the required EXI grammars




On Sat, Mar 16, 2013 at 1:56 AM, Peter Waher <Peter.Waher@clayster.com<mailto:Peter.Waher@clayster.com>> wrote:
Hello Taki

Thanks for your valuable input. You’re of course correct, so I’ve corrected §3.2 accordingly. I’ve attached the latest version.

Sincerely,
Peter Waher


From: Takuki Kamiya [mailto:tkamiya@us.fujitsu.com<mailto:tkamiya@us.fujitsu.com>]
Sent: den 15 mars 2013 21:26
To: Peter Waher

Cc: FABLET Youenn (Youenn.Fablet@crf.canon.fr<mailto:Youenn.Fablet@crf.canon.fr>); Joachim Lindborg; John Schneider; mact-usa@att.net<mailto:mact-usa@att.net>; Peter Saint-Andre; public-exi@w3.org<mailto:public-exi@w3.org>; Stephen Williams <sdw@lig.net<mailto:sdw@lig.net>> (sdw@lig.net<mailto:sdw@lig.net>); XMPP Standards; Yusuke DOI
Subject: RE: Proposal for including EXI in XMPP

Hi Peter,

Each EXI Body needs to always start with SD and ends with ED. SD is ethereal
(i.e. zero-bit), so its presence is indiscernible. On the other hand, ED often
carries bits (depending on EXI preservation settings in effect). Therefore,
stripping EXI Body grammar of them would amount to an alteration to the EXI
specification, which I think we should avoid. I suggest to adopt EXI body as a
whole.

Exerpted from the first paragraph of Section 3.2:
> The transmission of EXI-compressed stanzas takes the form of a sequence
> of EXI bodies. In order for the recipient to be able to correctly interpret
> these incoming EXI bodies, the sender is required to flush any pending bits
> at the end of the last End Element (EE) event for each stanza and then send
> any pending bytes available in the output buffer. Since this makes sure
> each EXI body starts at an even byte boundary, it permits the recipient to
> decompress the body into an XML stanza.

Assuming that each stanza is represented as an EXI body, it is the ED event
(instead of EE) for which flush operation needs to occur.

Regards,

taki


From: Peter Waher [mailto:Peter.Waher@clayster.com]
Sent: Friday, March 15, 2013 8:00 AM
To: Takuki Kamiya
Cc: FABLET Youenn (Youenn.Fablet@crf.canon.fr<mailto:Youenn.Fablet@crf.canon.fr>); Joachim Lindborg; John Schneider; mact-usa@att.net<mailto:mact-usa@att.net>; Peter Saint-Andre; public-exi@w3.org<mailto:public-exi@w3.org>; Stephen Williams <sdw@lig.net<mailto:sdw@lig.net>> (sdw@lig.net<mailto:sdw@lig.net>); XMPP Standards; Yusuke DOI
Subject: RE: Proposal for including EXI in XMPP

Hello Takuki,

Thank you for your valuable comments. I’ve rewritten §3.2 according to the ideas you presented. I also specify when the stream ends in the same section.

Would this address your comments?

I’ve attached the most recent revision.

Sincerely,
Peter Waher


From: Takuki Kamiya [mailto:tkamiya@us.fujitsu.com]
Sent: den 15 mars 2013 04:00
To: Peter Waher; public-exi@w3.org<mailto:public-exi@w3.org>
Cc: Joachim Lindborg (joachim.lindborg@sust.se<mailto:joachim.lindborg@sust.se>)
Subject: RE: Proposal for including EXI in XMPP

Hi Peter,

Thank you for sharing your XMPP work which already appear
to have collated many relevant points in producing an excellent
draft specification.

Section 3.2 in the updated version of the document describes
successive, back-to-back use of multiple EXI bodies. Since
this usage is something EXI does not directly describe, you might
want to make sure EXI bodies (except for the first EXI body
which begins immediately after a EXI header) each start at a byte
boundary by explicitly mentioning that is the case.

Also, you may want to use the terminology “EXI body” explicitly
in order to avoid each sequence of (SD ... ED) to be understood
as an EXI document. EXI 1.0 requires string table content to be
reset for each EXI document, which is in conflict with what I think
you want to achieve.

One thing that was not very clear to me was the way the recipient
recognizes that there is no more EXI body following one. Is there
going to be a stanza that represents it is the last one in the
communication?

Regards,

taki



From: Peter Waher [mailto:Peter.Waher@clayster.com]
Sent: Wednesday, March 13, 2013 9:43 AM
To: public-exi@w3.org<mailto:public-exi@w3.org>
Cc: Joachim Lindborg (joachim.lindborg@sust.se<mailto:joachim.lindborg@sust.se>)
Subject: Proposal for including EXI in XMPP

Hello

We have made a first draft of a proposal for incorporating EXI into XMPP networks. (See attached files.)

Anybody with an interest in EXI & XMPP are welcome to join us in this effort, please contact us. Any comments, suggestions, etc., on the contents of these documents are warmly welcomed and appreciated.

Sincerely,
Peter Waher