Re: Comparison of compression algorithms

As mentioned, asking around and was encouraged by a colleague to look
at JSON compression schemes.

https://www.lucidchart.com/techblog/2019/12/06/json-compression-alternative-binary-formats-and-compression-methods/

On Tue, 2020-09-01 at 15:10 -0400, Ted Guild wrote:
> On Tue, 2020-09-01 at 11:28 -0400, Ted Guild wrote:
> > We can/should also explore alternate formats Gunnar suggested
> > 
> > https://en.wikipedia.org/wiki/Apache_Avro
> > https://en.wikipedia.org/wiki/Protocol_Buffers
> > 
> > As these messages being transmitted are fairly small to begin with
> > and 
> > in-vehicle use case will have extremely low latency, I also want to
> > try
> > to understand how/where this would be more useful as
> > encoding/compressing and decoding will cost time and depending on
> > method non-negligible cpu. If the client app is just sampling to
> > offboard and won't unpack (decode), then we probably should look at
> > Extended Vehicle and other solutions being used for off-boarding in
> > addition to formats. What problem[s] are we trying to solve here?
> 
> We confirmed on the call today the primary use case is for off
> boarding
> data.
> 
> For a variety of reasons (security, connectivity, car being off),
> data
> being off-boarded from vehicle to the cloud will be pushed, not
> pulled.
> 
> Gen2 server instance can reside either in-vehicle or on a server in
> the
> cloud. 
> 
> For in-vehicle client apps that will be residing on the vehicle or
> nearby (local network) trusted devices, compression is not needed.
> Cloud servers will not be permitted to connect to Gen2 (ports not
> open)
> on a vehicle to make pull requests. Gen2 supports clients making
> pulls,
> HTTP GET and subscribe on Web Socket, it cannot initiate a push to
> server in the cloud.
> 
> Gen2 supporting alternate, including binary, formats besides JSON for
> more efficient local storage until it can send buffered data off the
> vehicle would help with 'every byte counts' when sending data off the
> vehicle.
> 
> VSS in protobuf format can make a ton of sense, perhaps with some
> further optimization along lines of what Ulf and Sanjeev have been
> working on.
> 
> Gen2 server residing in the cloud, exposing data already off-boarded
> can respond to pull requests from client apps running elsewhere in
> the
> cloud. Compression makes sense there.
> 
> If someone has a different architecture in mind where vehicle<->cloud
> connections differ fundamentally or we can translate a pull to push
> by
> redirecting and caching/buffering data, I would like to hear it.
> 
-- 
Ted Guild <ted@w3.org>
W3C Automotive Lead
https://www.w3.org/auto