Re: Redirection to Other IP Addresses

Bin, I'm still not sure if a major part of your use-case is to use
different Alternative Services for different URLs within the same Origin?
That seems like an important input requirement differentiating between a
number of the different options here.  It's also the one that changes the
semantics of Alt-Svc most dramatically over what they are today.  In
addition to being optional, Alt-Svc is also scoped to the Origin which
presumably simplifies both client implementation but also significantly
simplifies semantics.

For example, if a client is requesting a bunch of URLs in parallel from the
same origin
(eg, different large streaming movies), would we want them each to be able
to go to different places?  That seems like a major use-case not covered by
the current Alt-Svc.

A precursor to Alt-Svc covered a bunch of these, although the semantics
were not as fleshed out as they could have been:
draft-nygren-httpbis-connection-redirect-00
That both had a way to scope Alt-Svc to a Path as well as an attribute
indicating that this should be handled immediately:

   2.  The Path specified in the Alt-Server-Policy is a prefix of the
       request-path for the subsequent request (including the case where
       they string-compare equal).
   [...]
   Depending on the policy-when-value the behavior should change:
      * "immediate" - the client SHOULD abort this HTTP request and re-
         issue the request to an alt-server.  This MUST NOT be returned in
         response to non-idempotent requests (such as POST/PUT).
      * "next" = the client SHOULD continue with any in-flight HTTP
         requests but should in-parallel establish a connection to the alt-
         server for use in subsequent requests.
The impact on client complexity was a big reason these were abandoned
in Alt-Svc.

To Mark's point, Alt-Svc also hasn't gotten critical mass of good
client implementations yet, even so.
Handling clients not supporting this was also a challenge (ie, even
with the "immediate" label
as a hint it was less clear what to do about the content itself, such
as whether to return it,
do a 100 response, or pause some for clients to disconnect).

Extrapolating on Mike's OOB encoding suggestion, what about allowing
the Alternative-Service to be provided as a hint along with the OOB
response and scoped to that particular OOB-encoded URL?

Somewhere back in time or in some mail thread I had a proposal which
did this for this purpose.

For example, if the client signaled support through a header and the
response was:

HTTP/1.1 200 OK
Content-Encoding: out-of-band

Vary: Accept-Encoding

{
    "sr": [
      { "r" : "/c/bae27c36-fa6a-11e4-ae5d-00059a3c7a00",
               "alt-svc": [ "h2=\"cdn-node-75.example.net:443\"" ] }
    ]
}

Where in this case the alt-svc was scoped purely to the particular
OOB-encoded URL response.
In the case where the origin matches the origin returning the OOB
response, cookies can be sent.

All of these do add lots of complexity to client connection pool management
as connections no longer
can be keyed purely off of Origin but also need some way to index into some
Alt-Svc or path or URL property as well.
The impact on code complexity for connection management (and a need to
refactor) is a major reason why
one major client never added Alt-Svc support.  Even more complexity here
might make it less likely?

Overall this is a direction that I suspect that a number of
server-side / CDN providers would be quite interested in.
What would it take to get general-purpose client software interested?
Are any here interested? Without them it is an academic exercise.

Another reason to combine with OOB-encoding is that there are enough
other things that can be done with it (especially in cases
where cookies are NOT send and where the server CDN serving the
content is not trusted to see cookies or even the content itself)
to make clients potentially more interested.

       Erik



On Thu, Aug 22, 2019 at 12:18 AM Mark Nottingham <mnot@mnot.net> wrote:

> My .02 -
>
> Any new protocol element along these lines needs to be optional, to make
> it backwards-compatible with existing clients.
>
> Introducing a new status code here is application-visible, when you really
> want behaviour that is hidden. That's one of the big advantages of Alt-Svc,
> I think.
>
> So, a flag on Alt-Svc that basically says "I really want you to use this
> alt-svc for any new requests, even if you have to wait for that connection
> to spin up" would help.
>
> What you'd also need is a way to convey that information to the client for
> *this* response (since the Alt-Svc header only affects requests after the
> response it occurs upon).
>
> 421 Misdirected request would do that, but it means that clients that
> don't support 421 or Alt-Svc with a different hostname won't be able to get
> anything for that URL (see above: re optional).
>
> Modifying the semantics of the new Alt-Svc flag to "I really want you to
> use this alt-svc for *this* response and any new requests, even if you have
> to wait for that connection to spin up" might do it.
>
> A 1xx response that contains the Alt-Svc + new flag, followed by a short
> pause is kind of interesting here; it gives the client the opportunity to
> sever the connection without wasting (much) bandwidth.
>
> Or you could just put the Alt-Svc + new flag onto the normal response, and
> pace the start of the response a bit (for ~1RT) to a similar effect.
>
> The biggest barrier to adoption here by far is getting clients to adopt
> something; Alt-Svc was specified some time ago now, and is still not
> available cross-host in Chrome. The other thing to look at is how much
> surface area you're adding -- and therefore, how much security analysis is
> necessary. Reusing Alt-Svc seems like a good start there.
>
> Cheers,
>
>
> > On 15 Aug 2019, at 10:59 pm, craigt <c@gryning.com> wrote:
> >
> > Just catching up on this thread:
> >
> > Bin, It's great to hear of others wanting a solution in this area....
> >
> > I set up a proof of concept for something very similar to this using
> > Alt-Svc, and it worked really well (on Firefox). The preferred path
> > was almost always set up before the bulk download was initiated due to
> > companion assets. The problem was that most user agents only
> > implemented protocol upgrade via Alt-Svc rather than 'connection
> > migration' or almost in this case 'http traffic engineering'.
> > Alt-Svc also has other useful properties when used in conjunction with
> > single address endpoints like failback to origin and multiple
> > alternates.
> >
> > The use case described here, which was very similar to my own at the
> > time, has a 'lowest common capability' issue so the OPTIONAL aspects
> > of Alt-Svc make it unusable without support from the bulk of clients.
> > However, given Alt-Svc has become near essential for protocol upgrade
> > and partially supported by all, how can we encourage client
> > implementors to work towards:
> >
> > "Therefore, if a client supporting this specification becomes aware of
> >   an alternative service, the client SHOULD use that alternative
> >   service for all requests to the associated origin as soon as it is
> >   available"
> >
> > Or a method of signalling that Alt-Svc/equivalent is not optional in
> $case....
> >
> >
> > On Wed, 14 Aug 2019 at 23:53, Bin Ni <nibin@quantil.com> wrote:
> >>
> >> Hi Erik and All,
> >>
> >> My use case is we have a bunch of reverse proxy servers all over the
> world (just like any other CDN providers).
> >> Our customers deploy proxy policies to all those servers.
> >> Assume company ABC is one of our customers, they may configure all our
> servers to serve https://download.abc.com, with the certificate for
> "download..abc.com" installed on each of the servers.
> >> They also configure their DNS to point www.abc.com to a CNAME provided
> by us, such that my company essentially takes over the resolution of that
> hostname.
> >> When an end-user is visiting download.abc.com, the hostname is
> resolved by our DNS server to one of the server's IP addresses.
> >> This IP address is chosen based on some algorithm. But this algorithm
> has some limitations which may not always return an IP that is optimal for
> the end user.
> >> When the end user reaches that IP to download some object, for example
> https://download.abc.com/movie.mp4 (potentially with some cookie
> targeting "download.abc.com"), an algorithm running on that server
> >> finds out that another server would provide better performance and want
> to redirect the end user to the new server to download the object.
> >> Right now what we can do is sending 30X redirection to the end user
> with something like:
> >> Location: https://{new IP}.mycompany.com/download.abc.com/movie.mp4
> >> As I mentioned in earlier posts, this current method breaks cookie and
> company ABC's client (maybe a mobile App) may refuse to be directed to a
> different domain.
> >>
> >> Hope this is a clear enough description of the use case. Please let me
> know of any questions.
> >> Thanks!
> >>
> >> Bin
> >>
> >> On Wed, Aug 14, 2019 at 1:11 PM Erik Nygren <erik+ietf@nygren.org>
> wrote:
> >>>
> >>> You get something in the middle ground.  The Origin doesn't change,
> but cookies don't get sent without extra work.
> >>> But you gain the ability to decouple the server certificate and trust
> and integrity from the Origin.
> >>> They do solve slightly different but heavily overlapping problems.
> >>>
> >>> It may make sense to take a step back and enumerate the set of
> problems and use-cases
> >>> that would be good to solve and see which ones there is interest in
> addressing, and then
> >>> finding a solution that covers this set.
> >>>
> >>> In order for any effort on this front to be worthwhile it will also
> require
> >>> getting interest and buy-in to collaborate and implement from both
> server
> >>> operators and client implementations, with the client implementations
> >>> being the one that is going to be especially critical to success.
> >>> (Even Alt-Svc has had limited client implementation to-date.)
> >>>
> >>>     Erik
> >>>
> >>>
> >>>
> >>>
> >>>
> >>>
> >>> On Wed, Aug 14, 2019 at 3:38 PM Bin Ni <nibin@quantil.com> wrote:
> >>>>
> >>>> Hi Mike and Eric,
> >>>>
> >>>> If looks to me that if I use oob encoding for my use case, it will be
> the same as 30X redirection.
> >>>> I won't be able to achieve my goal of "only change the IP address,
> everything else remain the same".
> >>>>
> >>>> Bin
> >>>>
> >>>>
> >>>> On Wed, Aug 14, 2019 at 7:34 AM Mike Bishop <mbishop@evequefou.be>
> wrote:
> >>>>>
> >>>>> I second this – given that you’re looking for a forced redirect, an
> out-of-band response coupled with an Alt-Svc to the other host should
> provide the right behavior.  The first request goes OOB and the client gets
> the payload from the alternate; the subsequent requests can be made
> directly to the alternative, since it also has a cert for the origin.
> >>>>>
> >>>>>
> >>>>>
> >>>>> From: Erik Nygren <erik+ietf@nygren.org>
> >>>>> Sent: Thursday, August 8, 2019 9:27 PM
> >>>>> To: Bin Ni <nibin@quantil.com>
> >>>>> Cc: W. Felix Handte <w@felixhandte.com>; HTTP Working Group <
> ietf-http-wg@w3.org>
> >>>>> Subject: Re: Redirection to Other IP Addresses
> >>>>>
> >>>>>
> >>>>>
> >>>>> Another directional approach for this use-case would be out-of-band
> content encoding.
> >>>>>
> >>>>> For example:
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> https://tools.ietf.org/id/draft-reschke-http-oob-encoding-12.html
> >>>>>
> >>>>>
> >>>>>
> >>>>> It is possible that this could cover at least some of the use-cases.
> >>>>>
> >>>>> It has the benefit that the client signals support (via
> Accept-Encoding).
> >>>>>
> >>>>> The target won't get cookies, but authenticators could be passed in
> the URL target.
> >>>>>
> >>>>> It also has some better properties for not having to fully trust the
> redirect target
> >>>>>
> >>>>> and not having to have the same cert on the redirect target.
> >>>>>
> >>>>>
> >>>>>
> >>>>>    Erik
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Thu, Aug 1, 2019 at 2:58 PM Bin Ni <nibin@quantil.com> wrote:
> >>>>>
> >>>>> Hi Felix,
> >>>>>
> >>>>>
> >>>>>
> >>>>> What you described is exactly what my company (you probably already
> figured out that I work for a CDN company)
> >>>>>
> >>>>> is providing to our customers today. The problems are:
> >>>>>
> >>>>> 1. In your example, the first host "cdn.com" is the CDN customer's
> hostname. They usually can't provide us with a "*.geo.cdn.com" wildcard
> cert. Some of them requires EV certificate, which does not even support
> wildcard.
> >>>>>
> >>>>> 2. Cookies are often targeting a specific hostname. We can't ask all
> our customer to change the business logic of their web application to make
> sure all cookies are targeting the entire domain.
> >>>>>
> >>>>>
> >>>>>
> >>>>> Hope this helps. Any more questions?
> >>>>>
> >>>>>
> >>>>>
> >>>>> Bin
> >>>>>
> >>>>>
> >>>>>
> >>>>> On Thu, Aug 1, 2019 at 9:37 AM W. Felix Handte <w@felixhandte.com>
> wrote:
> >>>>>
> >>>>> Bin,
> >>>>>
> >>>>> I've been following along on this discussion and it's still not
> clear to
> >>>>> me why 30X doesn't solve this use case. Take for example a request
> and
> >>>>> response as follows.
> >>>>>
> >>>>>   GET /large_file HTTP/1.1
> >>>>>   Host: cdn.com
> >>>>>
> >>>>> To which the server responds with
> >>>>>
> >>>>>   HTTP/1.1 307
> >>>>>   Location: https://singapore.geo.cdn.com/large_file
> >>>>>
> >>>>> Or even
> >>>>>
> >>>>>   HTTP/1.1 307
> >>>>>   Location: https://123_45_67_89.ip.cdn.com/large_file
> >>>>>
> >>>>> Maybe I'm missing something, but as I understand it, HTTPS and
> Cookies
> >>>>> should work with the above (assuming you have wildcard certs for
> >>>>> *.geo.cdn.com and/or *.ip.cdn.com, and have set your cookies with
> >>>>> domain=.cdn.com). And it otherwise seems to accomplish exactly your
> intent.
> >>>>>
> >>>>> Can you explain in a little more detail why you believe something
> along
> >>>>> those lines wouldn't solve your need?
> >>>>>
> >>>>> Thanks,
> >>>>> Felix
> >>>>>
> >>>>> On 8/1/19 6:12 AM, Bin Ni wrote:
> >>>>>> Hi Daniel,
> >>>>>>
> >>>>>> At high level, my proposal is in every other way the same as
> today's 30X
> >>>>>> redirection.
> >>>>>> With this in mind, the answer to your questions are:
> >>>>>> 1. In general, the alternate IP should only be used once for the
> next
> >>>>>> single request.
> >>>>>> But there is nothing to prevent the clients from remembering it,
> which
> >>>>>> is OK.
> >>>>>> Just like there is nothing to prevent a client to disregard the DNS
> TTL.
> >>>>>> They do it with their own risk.
> >>>>>> 2. This proposal is to fix some limitations of the 30X with
> Location header.
> >>>>>> Not very helpful to make it work together with the Location header.
> >>>>>> 3. We are not requiring every server and every client to support
> this
> >>>>>> proposal.
> >>>>>> For the ones who find it to be useful, the "extra burden" is a
> non-issue.
> >>>>>>
> >>>>>> Thanks!
> >>>>>>
> >>>>>> Bin
> >>>>>>
> >>>>>> On Thu, Aug 1, 2019 at 2:18 AM Daniel Stenberg <daniel@haxx.se
> >>>>>> <mailto:daniel@haxx.se>> wrote:
> >>>>>>
> >>>>>>    On Thu, 1 Aug 2019, Bin Ni wrote:
> >>>>>>
> >>>>>>> 2. my proposed behavior:
> >>>>>>> Client: Hi Server-1.1.1.1, can you send me the movie XXX?
> >>>>>>> Server-1.1.1.1: Sorry, I can't give you the movie, you need to
> >>>>>>    ask server
> >>>>>>> 2.2.2.2 for this movie.
> >>>>>>> Client: Hi Server-2.2.2.2, can you send me the movie XXX?
> >>>>>>> Server-2.2.2.2: Here is the movie.
> >>>>>>> (It then took 0.5 hours to deliver the movie, because
> >>>>>>    server-2.2.2.2 is
> >>>>>>> closer to the client, or less loaded)
> >>>>>>
> >>>>>>    If we for a moment play with the idea that we'd do something like
> >>>>>>    this, then I
> >>>>>>    think it should be aligned with and work together with Alt-Svc
> in a
> >>>>>>    better way
> >>>>>>    than what is currently proposed...
> >>>>>>
> >>>>>>    There's no max-age/TTL. For how long is the user-agent supposed
> to
> >>>>>>    consider
> >>>>>>    the alternative IP addresses as the only ones that the given
> origin
> >>>>>>    has?
> >>>>>>    Forever? Only for the next single connect (attempt)?
> >>>>>>
> >>>>>>    Are the alternative IPs supposed to be used for the entire
> origin or
> >>>>>>    for that
> >>>>>>    specific URI only?
> >>>>>>
> >>>>>>    A 3xx redirect without a Location: header? Wouldn't it make more
> >>>>>>    sense and
> >>>>>>    work more similar to existing 3xx redirects if it also sends a
> >>>>>>    Location:? Then
> >>>>>>    existing clients that don't understand 312 might have a higher
> >>>>>>    chance of at
> >>>>>>    least doing something sensible.
> >>>>>>
> >>>>>>    If a client gets this response and starts downloading huge
> content
> >>>>>>    from the
> >>>>>>    new IP and the client then opens a second connection to the
> origin
> >>>>>>    in a second
> >>>>>>    tab. Which IPs is that supposed to use? The original ones or the
> >>>>>>    redirected
> >>>>>>    ones?
> >>>>>>
> >>>>>>    Requring user-agent snooping for a server to figure out if the
> >>>>>>    feature works
> >>>>>>    or not is a totally broken idea and I think this detail needs to
> be
> >>>>>>    worked out
> >>>>>>    for this idea to be considered for real.
> >>>>>>
> >>>>>>    My personal preference is probably to add some sort of "urgency"
> >>>>>>    thing to
> >>>>>>    alt-svc instead of this 312 plus several headers, so that a
> client
> >>>>>>    can be told
> >>>>>>    that it should switch sooner rather than later.
> >>>>>>
> >>>>>>    --
> >>>>>>
> >>>>>>       / daniel.haxx.se <http://daniel.haxx.se>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> --
> >>>>>>
> >>>>>> Bin Ni
> >>>>>> VP of Engineering
> >>>>>>
> >>>>>> Quantil
> >>>>>>
> >>>>>> Connecting users with content...it's that simple.
> >>>>>>
> >>>>>> Office: +1-888-847-9851 <tel:(888)%20847-9851>
> >>>>>>
> >>>>>> Tweeter <https://twitter.com/Team_Quantil> Google Plus
> >>>>>> <https://plus.google.com/+Quantil_team/> Linked In
> >>>>>> <https://www.linkedin.com/company/quantil>
> >>>>>>
> >>>>>> The information contained in this email may be confidential and/or
> >>>>>> legally privileged. It has been sent for the sole use of the
> intended
> >>>>>> recipient(s). If the reader of this message is not an intended
> >>>>>> recipient, you are hereby notified that any unauthorized review,
> use,
> >>>>>> disclosure, dissemination, distribution, or copying of this
> >>>>>> communication, or any of its contents, is strictly prohibited. If
> you
> >>>>>> have received this communication in error, please reply to the
> sender
> >>>>>> and destroy all copies of the message. To contact us directly, send
> to
> >>>>>> QUANTIL, INC. at 1919 S Bascom Ave #600, Campbell, CA 95008
> >>>>>> <
> https://maps.google.com/?q=1919+S+Bascom+Ave+%23600,+Campbell,+CA+95008&entry=gmail&source=g
> >,
> >>>>>> or visit our website at www.quantil.com. <https://www.quantil.com/>
> >>>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> --
> >>>>>
> >>>>> Bin Ni
> >>>>> VP of Engineering
> >>>>>
> >>>>> Connecting users with content...it's that simple.
> >>>>>
> >>>>> Office: +1-888-847-9851
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>>> The information contained in this email may be confidential and/or
> legally privileged. It has been sent for the sole use of the intended
> recipient(s). If the reader of this message is not an intended recipient,
> you are hereby notified that any unauthorized review, use, disclosure,
> dissemination, distribution, or copying of this communication, or any of
> its contents, is strictly prohibited. If you have received this
> communication in error, please reply to the sender and destroy all copies
> of the message. To contact us directly, send to QUANTIL, INC. at 1919 S
> Bascom Ave #600, Campbell, CA 95008, or visit our website at
> www.quantil.com.
> >>>>>
> >>>>>
> >>>>
> >>>>
> >>>>
> >>>> --
> >>>>
> >>>> Bin Ni
> >>>> VP of Engineering
> >>>>
> >>>> Connecting users with content...it's that simple.
> >>>>
> >>>> Office: +1-888-847-9851
> >>>>
> >>>>
> >>>>
> >>>>
> >>>> The information contained in this email may be confidential and/or
> legally privileged. It has been sent for the sole use of the intended
> recipient(s). If the reader of this message is not an intended recipient,
> you are hereby notified that any unauthorized review, use, disclosure,
> dissemination, distribution, or copying of this communication, or any of
> its contents, is strictly prohibited. If you have received this
> communication in error, please reply to the sender and destroy all copies
> of the message. To contact us directly, send to QUANTIL, INC. at 1919 S
> Bascom Ave #600, Campbell, CA 95008, or visit our website at
> www.quantil.com.
> >>
> >>
> >>
> >> --
> >>
> >> Bin Ni
> >> VP of Engineering
> >>
> >> Connecting users with content...it's that simple.
> >>
> >> Office: +1-888-847-9851
> >>
> >>
> >>
> >>
> >> The information contained in this email may be confidential and/or
> legally privileged. It has been sent for the sole use of the intended
> recipient(s). If the reader of this message is not an intended recipient,
> you are hereby notified that any unauthorized review, use, disclosure,
> dissemination, distribution, or copying of this communication, or any of
> its contents, is strictly prohibited. If you have received this
> communication in error, please reply to the sender and destroy all copies
> of the message. To contact us directly, send to QUANTIL, INC. at 1919 S
> Bascom Ave #600, Campbell, CA 95008, or visit our website at
> www.quantil.com.
> >
> >
>
> --
> Mark Nottingham   https://www.mnot.net/
>
>