Re: Structured request headers deployment issues

On Mon, Jun 22, 2020 at 5:19 AM Mark Nottingham <mnot@mnot.net> wrote:

> > On 19 Jun 2020, at 5:03 pm, Mike West <mkwst@google.com> wrote:
> >> Personally, I think that outreach and waiting is the right approach; if
> browsers consistently send these headers, they'll adapt, and the numbers
> are still relatively small -- or at least small enough that it's not likely
> the numbers will be reduced if the syntax is changed (due to _other_ WAFs'
> opinions about what a "good" request is).
> >
> > The flip-side of this is that we break sites users rely upon. That's
> tough to do at scale. In this case in particular, I think I agree with the
> underlying assertion that we'd like this syntax to work, and it seems
> reasonable to me to roll it out to a small percentage of stable users. That
> said, if we end up breaking the internet for even a small percentage of
> users, it doesn't seem like a good idea to hold them hostage in the hopes
> of increasing pressure on WAF vendors.
>
> I don't think we disagree. The problem is that there is always going to be
> a small percentage that break, unless we severely limit the evolution of
> the Web (i.e., don't use new headers at all, especially given the data Yoav
> is bringing!), so the relevant question is *how much*.
>
> In other words, there's a theshold below which you're comfortable breaking
> a *few* sites, and those few sites presumably feel the pressure to change.
> Above that threshold, you're unwilling to cause too much damage, and the
> sites presumably feel comfortable in persisting their behaviour
> (acknowledging that there's some fuzziness on both sides).
>

I agree that we're not disagreeing. :)

> My expectation is that we'd roll this out to a small percentage of
> stable, see a spike in bug reports (especially from enterprise folks who
> are likely to have WAF deployments, unlikely to broadly deploy beta
> channels, and unlikely to choose to enable usage statistics or histograms),
> see a corresponding spike in error codes for top-level navigations, and
> roll it back.
> >
> > I think it might be reasonable to explore alternate spellings at the
> same time, perhaps with some A/B testing to evaluate how well each weaves
> its way through middleware.
>
> What's missing here are the ability to engage the WAF community, and time.
> Unlike Web sites, there's a relatively smaller number of WAF vendors out
> there, so it's not unreasonable to engage with them to bring the numbers
> down below the threshold. That will take time, especially since WAFs don't
> yet have auto-update capabilities on par with browsers, but I think it's
> worth doing.
>

For better or worse, it seems to me that this asymmetry in update
capability puts more onus on user agents to fit themselves to the world as
it exists, as they're likely to be able to move more quickly (or at all!).


> What I don't want to see is the Web become ossified on what a few WAFs
> happened to do in June 2020.
>

To this end, I wonder if some sort of GREASEd header would be reasonable
for user agents to start sending on some subset of requests. The user agent
client hints might end up accidentally serving as that header, but I don't
think Chromium's implementation even attempts to exercise all the bits and
pieces of https://tools.ietf.org/html/draft-ietf-httpbis-header-structure.

>> Related, we're also seeing more examples WAFs limiting how we can evolve
> the protocol (e.g., <https://github.com/coreruleset/coreruleset/pull/1777>).
> There's been a bit of background chatter about writing something about this
> and creating better communication with that community; I'm not sure what
> that will look like yet, but if anyone has ideas or is interested, please
> say so.
> >
> > I am interested!
>
> Great. I've heard from a few other folks too, so we'll try to figure out
> the appropriate venue for further discussion.
>
> Cheers,
>
>
> --
> Mark Nottingham   https://www.mnot.net/
>
>