Re: WiSH: A General Purpose Message Framing over Byte-Stream Oriented Wire Protocols (HTTP)

On Fri, 2016-11-25 at 13:12 +1100, Mark Nottingham wrote:
> Andy,
> 
> The "people managing it" are the IETF. "Let the community do it" is
> how the IETF works, with guidance from IETF leadership. I had a 

No... when ws for http/1 was hashed out, there were some adults in the
room who could make judgments about what was or wasn't "a consensus" in
a kinda-definitive way.  Several times these calls were important in
stopping the definition going around in endless circles... even so you
will recall there were like 80+ drafts of it.

That has gone away now.

> discussion with our Area Director about this topic in Seoul, and we
> agreed that the hybi list was the best place to talk about
> WebSockets.

That's not necessarily a problem, but is either of you guys who decided
that going to restore the ML to have the kind of management it had when
it spat out RFC6455?  Or if it's chicken-and-egg, since the list is
moribund and no evident interest today to point to, some threshold for
that happening?  If not as I said it feels like it's just getting
punted over the wall.  It's going back to Hybi but the management of
Hybi is gone and it's nobody's problem...

-Andy

> Cheers,
> 
> 
> > On 25 Nov. 2016, at 11:51 am, Andy Green <andy@warmcat.com> wrote:
> > 
> > On Fri, 2016-11-25 at 10:35 +1100, Mark Nottingham wrote:
> > > To clarify, we said that the future of WebSockets really isn't in
> > > scope for this WG; the proper venue for discussing that is:
> > >   https://www.ietf.org/mailman/listinfo/hybi
> > 
> > They kind of shut that down, the people managing it have gone away
> > 
> > https://mailarchive.ietf.org/arch/msg/hybi/YVBJTzJcvzytIY46KfIS8bch
> > f5E
> > 
> > the ML is still running but is very quiet.
> > 
> > WS itself seems to be in rude health out there.
> > 
> > > What *is* in-scope here is how (if at all) that protocol
> > > interacts
> > > with HTTP, including HTTP/2; there are several ways you could
> > > implement WebSockets over HTTP/2, and a few pitfalls in doing so
> > > that
> > > the people on this list will be able to give you feedback on.
> > 
> > It's unfortunate that wasn't considered part of defining HTTP/2, so
> > it
> > could be baked in.   The subject was certainly raised.  But I can
> > understand the desire to get the main business out of the door.
> > 
> > > However, it's hard to do that before there's agreement in the WS
> > > community about what the requirements are. Ideally, that
> > > community
> > > would bring a single proposal that has broad support here for
> > > review.
> > 
> > Any formal mechanism to manage that has gone away for hybi
> > AFAICT.  So
> > this "let the community do it" feels like a bit of a cop-out /
> > bullet
> > dodging.
> > 
> > At any rate I think the number of people interested in HTTP/2 WS is
> > still very low compared to the number of people interested in WS
> > client
> > and server generally in my experience.  But at some point that will
> > change, possibly suddenly.
> > 
> > -Andy
> > 
> > > Cheers,
> > > 
> > > 
> > > > On 25 Nov. 2016, at 5:39 am, Van Catha <vans554@gmail.com>
> > > > wrote:
> > > > 
> > > > Thanks for clarification. Unfortunate that so little attention
> > > > was
> > > > paid to this.  Looks like some of us will be on HTTP1.1 for a
> > > > long
> > > > time.
> > > > 
> > > > On Mon, Nov 21, 2016 at 11:14 PM, Takeshi Yoshino <tyoshino@goo
> > > > gle.
> > > > com> wrote:
> > > > Ah, no. Martin just warned us that we might face the same issue
> > > > that SSE faced.
> > > > 
> > > > Mark's suggestion is a separate thing. The co-chairs (Mark and
> > > > Patrick) said that this (WiSH) doesn't seems to be a topic that
> > > > should be discussed in the HTTP WG given the charter of the WG,
> > > > I
> > > > think.
> > > > 
> > > > On Sun, Nov 20, 2016 at 12:26 PM, Van Catha <vans554@gmail.com>
> > > > wrote:
> > > > I do not understand what this means.  Is the suggestion to
> > > > ignore
> > > > WiSH for now in favor of SSE?
> > > > 
> > > > On Fri, Nov 18, 2016 at 1:55 AM, Takeshi Yoshino <tyoshino@goog
> > > > le.c
> > > > om> wrote:
> > > > I'd like to share the feedback on WiSH from IETF 97.
> > > > 
> > > > ----
> > > > 
> > > > Due to limited time, I got just one on-site comment from Martin
> > > > about comparison with Server-sent event (EventSource).
> > > > 
> > > > As mentioned in the I-D, yes, this is kinda full-duplex SSE
> > > > with
> > > > the WS framing, and it might suffer from unexpected buffering
> > > > by
> > > > intermediaries if any as Martin said.
> > > > 
> > > > WiSH should work well for deployment with TLS only (possibly
> > > > with
> > > > some non-TLS part beyond server side front-end but are under
> > > > control of the service providers). Given the wide trend of
> > > > encouraging TLS and browser vendors' implementation status of
> > > > H2, I
> > > > think we should prioritize layering simplicity than taking care
> > > > of
> > > > gain of WiSH/H2/TCP + transparent proxy (with unexpected
> > > > buffering)
> > > > case. For H2-less TLS-less environment, we can just use the
> > > > WebSocket protocol.
> > > > 
> > > > There can still be some risk of MITM (trusted) proxy and
> > > > unexpected
> > > > buffering with AntiVirus/Firewall for deployment with TLS, but
> > > > other WebSocket/H2 mapping proposals also have issues of
> > > > possible
> > > > blocking, buffering, etc. WebSocket/TCP's handshake success
> > > > rate
> > > > for non-TLS port 80 was also not so good when it started
> > > > getting
> > > > deployed, and got improved gradually. I think the problems will
> > > > get
> > > > resolved once WiSH is accepted widely, and I believe the total
> > > > pain
> > > > and cost would be smaller.
> > > > 
> > > > ----
> > > > 
> > > > Mark suggested that we should find some other right place than
> > > > HTTP
> > > > WG. I'll discuss this with Mark. Maybe we'll consult the
> > > > DISPATCH
> > > > WG.
> > > > 
> > > > ----
> > > > 
> > > > Thanks everyone for the feedback.
> > > > 
> > > > Takeshi
> > > > 
> > > > On Thu, Nov 3, 2016 at 3:20 AM, Costin Manolache <costin@gmail.
> > > > com>
> > > > wrote:
> > > > Good timing -  http://httpwg.org/http-extensions/encryption-pre
> > > > view
> > > > .html is addressing my concerns for
> > > > webpush ( and general 'encrypted content' ), we're still
> > > > discussing
> > > > some details, but for this use
> > > > case metadata won't be needed.
> > > > 
> > > > Costin
> > > > 
> > > > 
> > > > On Tue, Nov 1, 2016 at 10:34 PM Takeshi Yoshino <tyoshino@googl
> > > > e.co
> > > > m> wrote:
> > > > On Mon, Oct 31, 2016 at 5:57 AM, Wenbo Zhu <wenboz@google.com>
> > > > wrote:
> > > > 
> > > > 
> > > > On Sun, Oct 30, 2016 at 10:25 AM, Costin Manolache <costin@gmai
> > > > l.co
> > > > m> wrote:
> > > > Thanks for the answer and pointers. From earlier responses, it
> > > > seems possible to use GET
> > > > or a non-web-stream request to would avoid the extra cost of
> > > > the
> > > > pre-flight.
> > > > 
> > > > 
> > > > Yeah, at least the Content-Type in the HTTP request gets
> > > > eliminated.
> > > >  
> > > > One more question/issue: in some cases it would be good to send
> > > > some
> > > > metadata (headers) along with binary frames. For example in
> > > > webpush
> > > > the content is an encrypted
> > > > blob, and needs headers for the key/salt. I would assume a lot
> > > > of
> > > > other 'binary' messages would
> > > > benefit if simple metadata could be sent along. Would it be
> > > > possible to use one of the reserved
> > > > bits for 'has metadata' and add some encoded headers ? I know
> > > > in
> > > > websocket they are intended 
> > > > for 'extensions', but 'headers' seems a very common use case.
> > > > Q about webpush: is the metadata different for each binary
> > > > message? 
> > > > 
> > > > We discussed about metadata and how to use one of RSV bits etc.
> > > > For
> > > > the current version, let's make sure the WS compatibility is
> > > > fully
> > > > addressed (with minimum wire encoding like WiSH)
> > > > 
> > > > Agreed. Let's discuss the metadata needs separately. I agree
> > > > it's
> > > > important.
> > > >  
> > > >  
> > > > 
> > > > Having the binary frame use some MIME encoding to pass both
> > > > text
> > > > headers and the binary blob
> > > > is possible - but has complexity and overhead.
> > > > OTOH, if the binary blob relies on text headers (metata) to be
> > > > useful, then you probably need define a dedicated MIME
> > > > encoding.
> > > > 
> > > >   
> > > > 
> > > > Costin
> > > > 
> > > > On Sun, Oct 30, 2016 at 5:27 AM Takeshi Yoshino <tyoshino@googl
> > > > e.co
> > > > m> wrote:
> > > > Thanks, Van, Costin.
> > > > 
> > > > On Sun, Oct 30, 2016 at 2:43 AM, Costin Manolache <costin@gmail
> > > > .com
> > > > > wrote:
> > > > 
> > > > Good point - websocket is widely deployed, including IoT - and
> > > > the
> > > > header is pretty easy to handle anyways.
> > > > +1.
> > > > 
> > > > One question: is this intended to be handled by browsers, and
> > > > exposed using the W3C websocket API ?
> > > > Will a regular app be able to make WiSH requests and parse the
> > > > stream by itself, without browser
> > > > interference ? And if yes, any advice on how it interact with
> > > > CORS
> > > > ? 
> > > > 
> > > > The first step would be using Streams based upload/download via
> > > > the
> > > > Fetch API + protocol processing in JS.
> > > > 
> > > > The next step could be either introduction of an optimized
> > > > native
> > > > implementation of WiSH parser/framer in the form of the
> > > > TransformStream which can be used as follows:
> > > > 
> > > > const responsePromise = fetch(url, init);
> > > > responsePromise.then(response => {
> > > >   const wishStream =
> > > > response.body().pipeThrough(wishTransformStream);
> > > >   function readAndProcessMessage() {
> > > >     const readPromise = wishStream.read();
> > > >     readPromise.then(result => {
> > > >       if (result.done) {
> > > >         // End of stream.
> > > >         return;
> > > >       }
> > > > 
> > > >       const message = result.value;
> > > >       // Process the message
> > > >       // E.g. access message.opcode for opcode, message.body
> > > > for
> > > > the body data
> > > >       readAndProcessMessage();
> > > >     });
> > > >   }
> > > >   readAndProcessMessage();
> > > > });
> > > > 
> > > > and provide a polyfill that presents this as the WebSocket API,
> > > > and
> > > > (or skip the step and) go further i.e. native implementation
> > > > for
> > > > everything if it turns out optimization is critical.
> > > > 
> > > > We need to discuss this also in W3C/WHATWG.
> > > > 
> > > > Regarding CORS, if the request includes non CORS-safelisted
> > > > headers, fetch() based JS polyfills will be basically subject
> > > > to
> > > > the CORS preflight requirement. We could try to exempt some of
> > > > well
> > > > defined headers if any for CORS like WebSocket handshake's
> > > > headers
> > > > and server-sent event's Last-Event-Id are exempted. Regarding
> > > > the
> > > > proposed subprotocol negotiation in the form of combination of
> > > > the
> > > > Accept header and the Content-Type header, the Accept header is
> > > > one
> > > > of the CORS-safelisted headers, so it's not a problem. The
> > > > Content-
> > > > Type header is considered to be non-CORS-safelisted if it's
> > > > value
> > > > is none of the CORS-safelisted media types. So, WiSH media type
> > > > would trigger the preflight unless we exclude it.
> > > > 
> > > > Origin policy https://wicg.github.io/origin-policy/ might also
> > > > help.
> > > >  
> > > > 
> > > > Costin
> > > > 
> > > > On Fri, Oct 28, 2016 at 12:06 PM Takeshi Yoshino <tyoshino@goog
> > > > le.c
> > > > om> wrote:
> > > > Sorry for being ambivalent.
> > > > 
> > > > We can of course revisit each design decision we made for RFC
> > > > 6455
> > > > framing and search for the optimal again. But as:
> > > > - one of the main philosophies behind WiSH is compatibility
> > > > with
> > > > WebSocket in terms of both spec and implementation
> > > > - the WebSocket is widely deployed and therefore we have a lot
> > > > of
> > > > implementations in various languages/platform
> > > > - most browsers already have logic for the framing
> > > > - the framing is not considered to be so big pain
> > > > inheriting the WebSocket framing almost as-is is just good
> > > > enough.
> > > > Basically, I'm leaning toward this plan.
> > > > 
> > > > Takeshi
> > > > 
> > > > On Sat, Oct 29, 2016 at 3:12 AM, Takeshi Yoshino <tyoshino@goog
> > > > le.c
> > > > om> wrote:
> > > > On Sat, Oct 29, 2016 at 2:55 AM, Loïc Hoguin <essen@ninenines.e
> > > > u>
> > > > wrote:
> > > > On 10/28/2016 08:41 PM, Costin Manolache wrote:
> > > > Current overhead is 2 bytes if frame is up to 125 bytes long -
> > > > which I
> > > > think it's not very common,
> > > > 4 bytes for up to 64k, and 10 bytes for anything larger.
> > > > IMHO adding one byte - i.e. making it fixed 5-byte, with first
> > > > as
> > > > is,
> > > > and next 4 fixed length would
> > > > be easiest to parse.
> > > > 
> > > > Is making it easy (or easier) to parse even a concern anymore?
> > > > 
> > > > Considering the number of agents and servers already supporting
> > > > Websocket, the numerous libraries for nearly all languages and
> > > > the
> > > > great autobahntestsuite project validating it all, reusing the
> > > > existing code is a very sensible solution.
> > > > 
> > > > 
> > > > Yeah, I've been having similar feeling regarding cost for
> > > > parser/encoder implementation though I might be biased.
> > > >  
> > > > There are obviously too many options to encode and each has
> > > > benefits -
> > > > my only concern was
> > > > that the choice of 1, 2, 8 bytes for length may not match
> > > > common
> > > > sizes.
> > > > 
> > > > ( in webpush frames will be <4k ).
> > > > 
> > > > -- 
> > > > Loïc Hoguin
> > > > https://ninenines.eu
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > > 
> > > 
> > > --
> > > Mark Nottingham   https://www.mnot.net/
> > > 
> > > 
> > > 
> > > 
> > > 
> 
> --
> Mark Nottingham   https://www.mnot.net/
> 
> 
> 
> 
>

Received on Friday, 25 November 2016 02:32:42 UTC