AW: [TF-TD] encoding formats

Hi Dave,

Thank you for all your work!

> I am in the midst of implementing a Web of Things server in C++ for microcontrollers
> like the Arduino where memory and packet size are both very limited.  I am therefore
> looking at ways to use more compact encodings for the messages used between a
> proxy and the thing it proxies.  In the NodeJS server I’ve started with JSON over
> Web Sockets, and now want to add support for CoAP and MQTT.
>
> For the microcontroller, I will need to use compact message formats as a means to
> achieve short messages, and to avoid the memory overhead of JSON. There is a very
> small Arduino library for CBOR, but I couldn’t find one for EXI.

There are several EXI implementations available, see [1] (also for C/C++). EXIP [2] is the only project, I am aware of, which is free and open source. Nevertheless, from my experience I can tell there are many more available for internal use.

> Given that the server hosting the proxy, and the server hosting the thing being
> proxied, both have copies of the thing’s model, it should be possible to use the
> model as a shared symbol table with a view to avoiding the need to send these
> as strings. I am therefore experimenting with a compact byte code format where
> symbols can be encoded with a single byte in common situations. I will give more
> details when it is all working.

Personally I think it is not sensible to come up with a "new" solution given the fact that we do have already solutions in place.
Note: Besides XML, the EXI working group is also working on supporting JSON.

Also, no matter how "simple" an approach starts people wish to add more features and things start to get complex.

> It would be interesting to discuss design issues around message encoding formats,
> e.g. how much memory is needed to support the encoding and decoding, how much
> CPU effort is needed, and how compact the formats are.

Yes, definitely!
Nevertheless, from the actual work that would be needed we are at best going to touch the surface. The EXI working group spend a lot of time doing this for XML only!

> p.s. I am not advocating that the Web of Things have a single encoding format,
> as I think the choice will depend on the context.  Of course between any pair of
> servers, they will need to find common protocols, data formats and encodings. If that
> can’t be done, then in principle, an intermediary could be used to bridge the differences.

I agree with you and that's why I believe we should NOT invent yet another encoding format.

Best,

-- Daniel

[1] http://www.w3.org/XML/EXI/#implementations
[2] exip.sourceforge.net/

Received on Tuesday, 30 June 2015 14:20:14 UTC