FW: EXI proposal objections/responses

Fyi

It might be of interest to you that the XSF council has accepted the EXI XEP as an experimental XEP (first stage). During the meeting several concerns were raised. Some are interesting, so I forward my responses to these concerns to you all. Any comments are appreciated.

Sincerely,
Peter Waher


From: Peter Waher
Sent: den 20 mars 2013 19:18
To: standards@xmpp.org
Subject: EXI proposal objections/responses

Hello

I tried to take part of the XSF Council meeting today, if questions would arise regarding the EXI or sensor data proposals, but for some reason I was disconnected and was later unable to reconnect.

However, from the XSF Council log (http://logs.xmpp.org/council/130320/) I see that there were some concerns and questions which I would have liked to respond to, perhaps to make it clearer why certain choices have been made.

Following questions/objections arose:


*         If schema negotiation and schema upload can be handled pre-authenticate or not:

I would say that it can, and it perhaps also should, for reasons clearer below (last item). The server can always put a limit on number of schemas uploaded and/or remove them (or not persist them) if authentication is not successful later. An implementation note will be added regarding this.



*         Several commented that one should not compress after encryption:



This is a correct statement. This is why compression in this XEP occurs before encryption.



*         Somebody showed concern regarding downloading certificates from the server:



This is not included in the proposal. The only party (possibly) receiving new schemas is the server. And only if the server agrees.



*         Concern arose about possibility to inject a schema that fundamentally changes data:



This is correct to the point where an invalid schema changes compression and decompression. What would result is garbage in one end or the other. That's the reason for the MD5 Hash attribute. To make sure both parties use exactly the same schema file.



*         Somebody also showed the concern that an injected schema could cause nasty issues:



It's good to vent such issues, and security experts should analyze this so everybody is contented. As I see it, there are two options:



1)      An injected schema change compression/decompression on the server side by error. In this case the result would be garbage in one end or the other.

2)      An injected schema is maliciously composed so that it decompresses something entirely different on the server side. But, in this case, the client who injected the schema could as well have sent that other XML in the first place.

One thing that the server should look out for (and I'll update the proposal with this, to make it clearer), is that a server should not accept a new schema with a given Hash if one already exists. In this way, the server protects itself from malicious clients trying to destroy or change existing compression mechanisms already in place.

Another step (which I'll also add) is for the client to be able to download a schema. In this way, it will be possible to detect if somebody has preinstalled a schema that is incorrect, before a proper client has the chance to install a correct version.

It's important to note that this possibility exists, because of the desire to be able to upload schemas that are EXI-compressed. The original proposal did not have this flaw, since it left hash and byte length calculation to the server, and a binary exact copy of the schema was transferred or downloaded.

One way to circumvent this problem, is to allow an option at the server side to only persist schemas if an operator of the server manually accepts the proposed schemas. It would work in sensor networks, where one can accept schemas during an installation phase, and then reject new proposed schemas in a production phase.


*         Many seem to think that a pure binary binding seems the correct path to go:



I've heard this from many people. And it's true that in some cases it might be easier to do this (at least for developers). However, it misses one of the main reasons why the XEP-0138 path was chosen: To allow for installation in any IT environment where XMPP is allowed, you need to go through port 5222. This is an accepted port number, it's accepted by IT departments, existing firewalls, etc. So, to be able to create applications that will be accepted by any IT department (accepting XMPP) and existing infrastructure (accepting XMPP), it has to go through this port, and consequentially, adapt itself to the requirements imposed by the XMPP core specification.



That being said, I'm all in favor for a parallel proposal being worked out, using a binary-only port. Once a port number has been assigned, and IETF has standardized its use, it will be a serious option. (Otherwise, the solution will be seen as just another proprietary protocol not based on standards - even though some parts are.) But before the IETF has agreed and proposed a standard regarding this, it will have to be a secondary option.



*         Some concerns have been raised that this proposal has slight variations from regular EXI:



I would not say so. The compression/decompression part is the same. EXI options are the same, except a few that have been removed since they do not work well together with XMPP. Also, an XMPP-specific (relating to how the XMPP server should use the EXI compression/decompression engine) option has been added (which the server can reject if it wants to).



*         It has also been raised that implementing EXI-compression using XEP-0138 would require small devices with limited memory to implement a complete XML parsing stack:



I don't see that. If we compare with other solutions that such devices already implement (for example ZigBee), they communicate with EXI over REST. Is the device required to implement the entire HTTP (1.1) protocol thus creating a full-fledged HTTP server? Of course this is not true. What the device needs to do, is being able to send strings and extract information from strings. It might be a minute difference between parsing an HTTP header and parsing incoming XML fragments having a predefined format, but the difference is not so great as to reject the idea and loose the possibility of using port 5222.



This is also the main reason why EXI negotiation should occur as the first this a client does after connecting, to make initial XML parsing as light as possible, without the need to implement a complete XML DOM.

Sincerely,
Peter Waher