Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)

Good timing -  http://httpwg.org/http-extensions/encryption-preview.html is
addressing my concerns for
webpush ( and general 'encrypted content' ), we're still discussing some
details, but for this use
case metadata won't be needed.

Costin


On Tue, Nov 1, 2016 at 10:34 PM Takeshi Yoshino <tyoshino@google.com> wrote:

> On Mon, Oct 31, 2016 at 5:57 AM, Wenbo Zhu <wenboz@google.com> wrote:
>
>
>
> On Sun, Oct 30, 2016 at 10:25 AM, Costin Manolache <costin@gmail.com>
> wrote:
>
> Thanks for the answer and pointers. From earlier responses, it seems
> possible to use GET
> or a non-web-stream request to would avoid the extra cost of the
> pre-flight.
>
>
>
> Yeah, at least the Content-Type in the HTTP request gets eliminated.
>
>
> One more question/issue: in some cases it would be good to send some
> metadata (headers) along with binary frames. For example in webpush the
> content is an encrypted
> blob, and needs headers for the key/salt. I would assume a lot of other
> 'binary' messages would
> benefit if simple metadata could be sent along. Would it be possible to
> use one of the reserved
> bits for 'has metadata' and add some encoded headers ? I know in websocket
> they are intended
> for 'extensions', but 'headers' seems a very common use case.
>
> Q about webpush: is the metadata different for each binary message?
>
> We discussed about metadata and how to use one of RSV bits etc. For the
> current version, let's make sure the WS compatibility is fully addressed
> (with minimum wire encoding like WiSH)
>
>
> Agreed. Let's discuss the metadata needs separately. I agree it's
> important.
>
>
>
>
>
> Having the binary frame use some MIME encoding to pass both text headers
> and the binary blob
> is possible - but has complexity and overhead.
>
> OTOH, if the binary blob relies on text headers (metata) to be useful,
> then you probably need define a dedicated MIME encoding.
>
>
>
>
> Costin
>
> On Sun, Oct 30, 2016 at 5:27 AM Takeshi Yoshino <tyoshino@google.com>
> wrote:
>
> Thanks, Van, Costin.
>
> On Sun, Oct 30, 2016 at 2:43 AM, Costin Manolache <costin@gmail.com>
> wrote:
>
> Good point - websocket is widely deployed, including IoT - and the header
> is pretty easy to handle anyways.
> +1.
>
> One question: is this intended to be handled by browsers, and exposed
> using the W3C websocket API ?
> Will a regular app be able to make WiSH requests and parse the stream by
> itself, without browser
> interference ? And if yes, any advice on how it interact with CORS ?
>
>
> The first step would be using Streams based upload/download via the Fetch
> API + protocol processing in JS.
>
> The next step could be either introduction of an optimized native
> implementation of WiSH parser/framer in the form of the TransformStream
> which can be used as follows:
>
> const responsePromise = fetch(url, init);
> responsePromise.then(response => {
>   const wishStream = response.body().pipeThrough(wishTransformStream);
>   function readAndProcessMessage() {
>     const readPromise = wishStream.read();
>     readPromise.then(result => {
>       if (result.done) {
>         // End of stream.
>         return;
>       }
>
>       const message = result.value;
>       // Process the message
>       // E.g. access message.opcode for opcode, message.body for the body
> data
>       readAndProcessMessage();
>     });
>   }
>   readAndProcessMessage();
> });
>
> and provide a polyfill that presents this as the WebSocket API, and (or
> skip the step and) go further i.e. native implementation for everything if
> it turns out optimization is critical.
>
> We need to discuss this also in W3C/WHATWG.
>
> Regarding CORS, if the request includes non CORS-safelisted headers,
> fetch() based JS polyfills will be basically subject to the CORS preflight
> requirement. We could try to exempt some of well defined headers if any for
> CORS like WebSocket handshake's headers and server-sent event's
> Last-Event-Id are exempted. Regarding the proposed subprotocol negotiation
> in the form of combination of the Accept header and the Content-Type
> header, the Accept header is one of the CORS-safelisted headers, so it's
> not a problem. The Content-Type header is considered to be
> non-CORS-safelisted if it's value is none of the CORS-safelisted media
> types. So, WiSH media type would trigger the preflight unless we exclude it.
>
> Origin policy https://wicg.github.io/origin-policy/ might also help.
>
>
>
> Costin
>
> On Fri, Oct 28, 2016 at 12:06 PM Takeshi Yoshino <tyoshino@google.com>
> wrote:
>
> Sorry for being ambivalent.
>
> We can of course revisit each design decision we made for RFC 6455 framing
> and search for the optimal again. But as:
> - one of the main philosophies behind WiSH is compatibility with WebSocket
> in terms of both spec and implementation
> - the WebSocket is widely deployed and therefore we have a lot of
> implementations in various languages/platform
> - most browsers already have logic for the framing
> - the framing is not considered to be so big pain
> inheriting the WebSocket framing almost as-is is just good enough.
> Basically, I'm leaning toward this plan.
>
> Takeshi
>
> On Sat, Oct 29, 2016 at 3:12 AM, Takeshi Yoshino <tyoshino@google.com>
> wrote:
>
> On Sat, Oct 29, 2016 at 2:55 AM, Loïc Hoguin <essen@ninenines.eu> wrote:
>
> On 10/28/2016 08:41 PM, Costin Manolache wrote:
>
> Current overhead is 2 bytes if frame is up to 125 bytes long - which I
> think it's not very common,
> 4 bytes for up to 64k, and 10 bytes for anything larger.
> IMHO adding one byte - i.e. making it fixed 5-byte, with first as is,
> and next 4 fixed length would
> be easiest to parse.
>
>
> Is making it easy (or easier) to parse even a concern anymore?
>
> Considering the number of agents and servers already supporting Websocket,
> the numerous libraries for nearly all languages and the great
> autobahntestsuite project validating it all, reusing the existing code is a
> very sensible solution.
>
>
> Yeah, I've been having similar feeling regarding cost for parser/encoder
> implementation though I might be biased.
>
>
> There are obviously too many options to encode and each has benefits -
> my only concern was
> that the choice of 1, 2, 8 bytes for length may not match common sizes.
>
> ( in webpush frames will be <4k ).
>
>
> --
> Loïc Hoguin
> https://ninenines.eu
>
>
>
>
>