Re: Redirection to Other IP Addresses

Hi Erik,

Thanks for sharing, I wasn't aware of
draft-nygren-httpbis-connection-redirect-00 or some of the background to
Alt-Svc. I think there are some attractive properties to being able to
direct based on path but acknowledge the complexities.

One use case I worked on was to do layer 7 direction of adaptive video
streaming clients to different IPs based on the quality of the video [1].
For example, when a client requests a single video segment at HD quality,
all future requests for that quality should go elsewhere. We built a PoC
based on Alt-Svc and it would have been preferable to keep the DASH URL
pattern that encodes the representation detail in the path. Instead,
because of Alt-Svc's origin-level redirection, we had to map this
information into the host, which has several downsides for larger
deployments. At the time, I penciled out what it might look like to add
path-based pattern matching to Alt-Svc, and I probably came up with a
similar answer to what you describe.

[1] draft-pardue-mcast-quic-05,which specifies how Alt-Svc can be used to
advertise the properties of QUIC packets and HTTP/3 payloads being sent to
multicast IP addresses.




On Thu, Aug 22, 2019 at 1:44 PM Erik Nygren <erik+ietf@nygren.org> wrote:

> Bin, I'm still not sure if a major part of your use-case is to use
> different Alternative Services for different URLs within the same Origin?
> That seems like an important input requirement differentiating between a
> number of the different options here.  It's also the one that changes the
> semantics of Alt-Svc most dramatically over what they are today.  In
> addition to being optional, Alt-Svc is also scoped to the Origin which
> presumably simplifies both client implementation but also significantly
> simplifies semantics.
>
> For example, if a client is requesting a bunch of URLs in parallel from
> the same origin
> (eg, different large streaming movies), would we want them each to be able
> to go to different places?  That seems like a major use-case not covered by
> the current Alt-Svc.
>
> A precursor to Alt-Svc covered a bunch of these, although the semantics
> were not as fleshed out as they could have been:
> draft-nygren-httpbis-connection-redirect-00
> That both had a way to scope Alt-Svc to a Path as well as an attribute
> indicating that this should be handled immediately:
>
>    2.  The Path specified in the Alt-Server-Policy is a prefix of the
>        request-path for the subsequent request (including the case where
>        they string-compare equal).
>    [...]
>    Depending on the policy-when-value the behavior should change:
>       * "immediate" - the client SHOULD abort this HTTP request and re-
>          issue the request to an alt-server.  This MUST NOT be returned in
>          response to non-idempotent requests (such as POST/PUT).
>       * "next" = the client SHOULD continue with any in-flight HTTP
>          requests but should in-parallel establish a connection to the alt-
>          server for use in subsequent requests.
> The impact on client complexity was a big reason these were abandoned in Alt-Svc.
>
> To Mark's point, Alt-Svc also hasn't gotten critical mass of good client implementations yet, even so.
> Handling clients not supporting this was also a challenge (ie, even with the "immediate" label
> as a hint it was less clear what to do about the content itself, such as whether to return it,
> do a 100 response, or pause some for clients to disconnect).
>
> Extrapolating on Mike's OOB encoding suggestion, what about allowing
> the Alternative-Service to be provided as a hint along with the OOB response and scoped to that particular OOB-encoded URL?
>
> Somewhere back in time or in some mail thread I had a proposal which did this for this purpose.
>
> For example, if the client signaled support through a header and the response was:
>
> HTTP/1.1 200 OK
> Content-Encoding: out-of-band
>
> Vary: Accept-Encoding
>
> {
>     "sr": [
>       { "r" : "/c/bae27c36-fa6a-11e4-ae5d-00059a3c7a00",
>                "alt-svc": [ "h2=\"cdn-node-75.example.net:443\"" ] }
>     ]
> }
>
> Where in this case the alt-svc was scoped purely to the particular OOB-encoded URL response.
> In the case where the origin matches the origin returning the OOB response, cookies can be sent.
>
> All of these do add lots of complexity to client connection pool
> management as connections no longer
> can be keyed purely off of Origin but also need some way to index into
> some Alt-Svc or path or URL property as well.
> The impact on code complexity for connection management (and a need to
> refactor) is a major reason why
> one major client never added Alt-Svc support.  Even more complexity here
> might make it less likely?
>
> Overall this is a direction that I suspect that a number of server-side / CDN providers would be quite interested in.
> What would it take to get general-purpose client software interested?  Are any here interested? Without them it is an academic exercise.
>
> Another reason to combine with OOB-encoding is that there are enough other things that can be done with it (especially in cases
> where cookies are NOT send and where the server CDN serving the content is not trusted to see cookies or even the content itself)
> to make clients potentially more interested.
>
>        Erik
>
>
>
> On Thu, Aug 22, 2019 at 12:18 AM Mark Nottingham <mnot@mnot.net> wrote:
>
>> My .02 -
>>
>> Any new protocol element along these lines needs to be optional, to make
>> it backwards-compatible with existing clients.
>>
>> Introducing a new status code here is application-visible, when you
>> really want behaviour that is hidden. That's one of the big advantages of
>> Alt-Svc, I think.
>>
>> So, a flag on Alt-Svc that basically says "I really want you to use this
>> alt-svc for any new requests, even if you have to wait for that connection
>> to spin up" would help.
>>
>> What you'd also need is a way to convey that information to the client
>> for *this* response (since the Alt-Svc header only affects requests after
>> the response it occurs upon).
>>
>> 421 Misdirected request would do that, but it means that clients that
>> don't support 421 or Alt-Svc with a different hostname won't be able to get
>> anything for that URL (see above: re optional).
>>
>> Modifying the semantics of the new Alt-Svc flag to "I really want you to
>> use this alt-svc for *this* response and any new requests, even if you have
>> to wait for that connection to spin up" might do it.
>>
>> A 1xx response that contains the Alt-Svc + new flag, followed by a short
>> pause is kind of interesting here; it gives the client the opportunity to
>> sever the connection without wasting (much) bandwidth.
>>
>> Or you could just put the Alt-Svc + new flag onto the normal response,
>> and pace the start of the response a bit (for ~1RT) to a similar effect.
>>
>> The biggest barrier to adoption here by far is getting clients to adopt
>> something; Alt-Svc was specified some time ago now, and is still not
>> available cross-host in Chrome. The other thing to look at is how much
>> surface area you're adding -- and therefore, how much security analysis is
>> necessary. Reusing Alt-Svc seems like a good start there.
>>
>> Cheers,
>>
>>
>> > On 15 Aug 2019, at 10:59 pm, craigt <c@gryning.com> wrote:
>> >
>> > Just catching up on this thread:
>> >
>> > Bin, It's great to hear of others wanting a solution in this area....
>> >
>> > I set up a proof of concept for something very similar to this using
>> > Alt-Svc, and it worked really well (on Firefox). The preferred path
>> > was almost always set up before the bulk download was initiated due to
>> > companion assets. The problem was that most user agents only
>> > implemented protocol upgrade via Alt-Svc rather than 'connection
>> > migration' or almost in this case 'http traffic engineering'.
>> > Alt-Svc also has other useful properties when used in conjunction with
>> > single address endpoints like failback to origin and multiple
>> > alternates.
>> >
>> > The use case described here, which was very similar to my own at the
>> > time, has a 'lowest common capability' issue so the OPTIONAL aspects
>> > of Alt-Svc make it unusable without support from the bulk of clients.
>> > However, given Alt-Svc has become near essential for protocol upgrade
>> > and partially supported by all, how can we encourage client
>> > implementors to work towards:
>> >
>> > "Therefore, if a client supporting this specification becomes aware of
>> >   an alternative service, the client SHOULD use that alternative
>> >   service for all requests to the associated origin as soon as it is
>> >   available"
>> >
>> > Or a method of signalling that Alt-Svc/equivalent is not optional in
>> $case....
>> >
>> >
>> > On Wed, 14 Aug 2019 at 23:53, Bin Ni <nibin@quantil.com> wrote:
>> >>
>> >> Hi Erik and All,
>> >>
>> >> My use case is we have a bunch of reverse proxy servers all over the
>> world (just like any other CDN providers).
>> >> Our customers deploy proxy policies to all those servers.
>> >> Assume company ABC is one of our customers, they may configure all our
>> servers to serve https://download.abc.com, with the certificate for
>> "download..abc.com" installed on each of the servers.
>> >> They also configure their DNS to point www.abc.com to a CNAME
>> provided by us, such that my company essentially takes over the resolution
>> of that hostname.
>> >> When an end-user is visiting download.abc.com, the hostname is
>> resolved by our DNS server to one of the server's IP addresses.
>> >> This IP address is chosen based on some algorithm. But this algorithm
>> has some limitations which may not always return an IP that is optimal for
>> the end user.
>> >> When the end user reaches that IP to download some object, for example
>> https://download.abc.com/movie.mp4 (potentially with some cookie
>> targeting "download.abc.com"), an algorithm running on that server
>> >> finds out that another server would provide better performance and
>> want to redirect the end user to the new server to download the object.
>> >> Right now what we can do is sending 30X redirection to the end user
>> with something like:
>> >> Location: https://{new IP}.mycompany.com/download.abc.com/movie.mp4
>> >> As I mentioned in earlier posts, this current method breaks cookie and
>> company ABC's client (maybe a mobile App) may refuse to be directed to a
>> different domain.
>> >>
>> >> Hope this is a clear enough description of the use case. Please let me
>> know of any questions.
>> >> Thanks!
>> >>
>> >> Bin
>> >>
>> >> On Wed, Aug 14, 2019 at 1:11 PM Erik Nygren <erik+ietf@nygren.org>
>> wrote:
>> >>>
>> >>> You get something in the middle ground.  The Origin doesn't change,
>> but cookies don't get sent without extra work.
>> >>> But you gain the ability to decouple the server certificate and trust
>> and integrity from the Origin.
>> >>> They do solve slightly different but heavily overlapping problems.
>> >>>
>> >>> It may make sense to take a step back and enumerate the set of
>> problems and use-cases
>> >>> that would be good to solve and see which ones there is interest in
>> addressing, and then
>> >>> finding a solution that covers this set.
>> >>>
>> >>> In order for any effort on this front to be worthwhile it will also
>> require
>> >>> getting interest and buy-in to collaborate and implement from both
>> server
>> >>> operators and client implementations, with the client implementations
>> >>> being the one that is going to be especially critical to success.
>> >>> (Even Alt-Svc has had limited client implementation to-date.)
>> >>>
>> >>>     Erik
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>>
>> >>> On Wed, Aug 14, 2019 at 3:38 PM Bin Ni <nibin@quantil.com> wrote:
>> >>>>
>> >>>> Hi Mike and Eric,
>> >>>>
>> >>>> If looks to me that if I use oob encoding for my use case, it will
>> be the same as 30X redirection.
>> >>>> I won't be able to achieve my goal of "only change the IP address,
>> everything else remain the same".
>> >>>>
>> >>>> Bin
>> >>>>
>> >>>>
>> >>>> On Wed, Aug 14, 2019 at 7:34 AM Mike Bishop <mbishop@evequefou.be>
>> wrote:
>> >>>>>
>> >>>>> I second this – given that you’re looking for a forced redirect, an
>> out-of-band response coupled with an Alt-Svc to the other host should
>> provide the right behavior.  The first request goes OOB and the client gets
>> the payload from the alternate; the subsequent requests can be made
>> directly to the alternative, since it also has a cert for the origin.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> From: Erik Nygren <erik+ietf@nygren.org>
>> >>>>> Sent: Thursday, August 8, 2019 9:27 PM
>> >>>>> To: Bin Ni <nibin@quantil.com>
>> >>>>> Cc: W. Felix Handte <w@felixhandte.com>; HTTP Working Group <
>> ietf-http-wg@w3.org>
>> >>>>> Subject: Re: Redirection to Other IP Addresses
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> Another directional approach for this use-case would be out-of-band
>> content encoding.
>> >>>>>
>> >>>>> For example:
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> https://tools.ietf.org/id/draft-reschke-http-oob-encoding-12.html
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> It is possible that this could cover at least some of the use-cases.
>> >>>>>
>> >>>>> It has the benefit that the client signals support (via
>> Accept-Encoding).
>> >>>>>
>> >>>>> The target won't get cookies, but authenticators could be passed in
>> the URL target.
>> >>>>>
>> >>>>> It also has some better properties for not having to fully trust
>> the redirect target
>> >>>>>
>> >>>>> and not having to have the same cert on the redirect target.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>    Erik
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Thu, Aug 1, 2019 at 2:58 PM Bin Ni <nibin@quantil.com> wrote:
>> >>>>>
>> >>>>> Hi Felix,
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> What you described is exactly what my company (you probably already
>> figured out that I work for a CDN company)
>> >>>>>
>> >>>>> is providing to our customers today. The problems are:
>> >>>>>
>> >>>>> 1. In your example, the first host "cdn.com" is the CDN customer's
>> hostname. They usually can't provide us with a "*.geo.cdn.com" wildcard
>> cert. Some of them requires EV certificate, which does not even support
>> wildcard.
>> >>>>>
>> >>>>> 2. Cookies are often targeting a specific hostname. We can't ask
>> all our customer to change the business logic of their web application to
>> make sure all cookies are targeting the entire domain.
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> Hope this helps. Any more questions?
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> Bin
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> On Thu, Aug 1, 2019 at 9:37 AM W. Felix Handte <w@felixhandte.com>
>> wrote:
>> >>>>>
>> >>>>> Bin,
>> >>>>>
>> >>>>> I've been following along on this discussion and it's still not
>> clear to
>> >>>>> me why 30X doesn't solve this use case. Take for example a request
>> and
>> >>>>> response as follows.
>> >>>>>
>> >>>>>   GET /large_file HTTP/1.1
>> >>>>>   Host: cdn.com
>> >>>>>
>> >>>>> To which the server responds with
>> >>>>>
>> >>>>>   HTTP/1.1 307
>> >>>>>   Location: https://singapore.geo.cdn.com/large_file
>> >>>>>
>> >>>>> Or even
>> >>>>>
>> >>>>>   HTTP/1.1 307
>> >>>>>   Location: https://123_45_67_89.ip.cdn.com/large_file
>> >>>>>
>> >>>>> Maybe I'm missing something, but as I understand it, HTTPS and
>> Cookies
>> >>>>> should work with the above (assuming you have wildcard certs for
>> >>>>> *.geo.cdn.com and/or *.ip.cdn.com, and have set your cookies with
>> >>>>> domain=.cdn.com). And it otherwise seems to accomplish exactly
>> your intent.
>> >>>>>
>> >>>>> Can you explain in a little more detail why you believe something
>> along
>> >>>>> those lines wouldn't solve your need?
>> >>>>>
>> >>>>> Thanks,
>> >>>>> Felix
>> >>>>>
>> >>>>> On 8/1/19 6:12 AM, Bin Ni wrote:
>> >>>>>> Hi Daniel,
>> >>>>>>
>> >>>>>> At high level, my proposal is in every other way the same as
>> today's 30X
>> >>>>>> redirection.
>> >>>>>> With this in mind, the answer to your questions are:
>> >>>>>> 1. In general, the alternate IP should only be used once for the
>> next
>> >>>>>> single request.
>> >>>>>> But there is nothing to prevent the clients from remembering it,
>> which
>> >>>>>> is OK.
>> >>>>>> Just like there is nothing to prevent a client to disregard the
>> DNS TTL.
>> >>>>>> They do it with their own risk.
>> >>>>>> 2. This proposal is to fix some limitations of the 30X with
>> Location header.
>> >>>>>> Not very helpful to make it work together with the Location header.
>> >>>>>> 3. We are not requiring every server and every client to support
>> this
>> >>>>>> proposal.
>> >>>>>> For the ones who find it to be useful, the "extra burden" is a
>> non-issue.
>> >>>>>>
>> >>>>>> Thanks!
>> >>>>>>
>> >>>>>> Bin
>> >>>>>>
>> >>>>>> On Thu, Aug 1, 2019 at 2:18 AM Daniel Stenberg <daniel@haxx.se
>> >>>>>> <mailto:daniel@haxx.se>> wrote:
>> >>>>>>
>> >>>>>>    On Thu, 1 Aug 2019, Bin Ni wrote:
>> >>>>>>
>> >>>>>>> 2. my proposed behavior:
>> >>>>>>> Client: Hi Server-1.1.1.1, can you send me the movie XXX?
>> >>>>>>> Server-1.1.1.1: Sorry, I can't give you the movie, you need to
>> >>>>>>    ask server
>> >>>>>>> 2.2.2.2 for this movie.
>> >>>>>>> Client: Hi Server-2.2.2.2, can you send me the movie XXX?
>> >>>>>>> Server-2.2.2.2: Here is the movie.
>> >>>>>>> (It then took 0.5 hours to deliver the movie, because
>> >>>>>>    server-2.2.2.2 is
>> >>>>>>> closer to the client, or less loaded)
>> >>>>>>
>> >>>>>>    If we for a moment play with the idea that we'd do something
>> like
>> >>>>>>    this, then I
>> >>>>>>    think it should be aligned with and work together with Alt-Svc
>> in a
>> >>>>>>    better way
>> >>>>>>    than what is currently proposed...
>> >>>>>>
>> >>>>>>    There's no max-age/TTL. For how long is the user-agent supposed
>> to
>> >>>>>>    consider
>> >>>>>>    the alternative IP addresses as the only ones that the given
>> origin
>> >>>>>>    has?
>> >>>>>>    Forever? Only for the next single connect (attempt)?
>> >>>>>>
>> >>>>>>    Are the alternative IPs supposed to be used for the entire
>> origin or
>> >>>>>>    for that
>> >>>>>>    specific URI only?
>> >>>>>>
>> >>>>>>    A 3xx redirect without a Location: header? Wouldn't it make more
>> >>>>>>    sense and
>> >>>>>>    work more similar to existing 3xx redirects if it also sends a
>> >>>>>>    Location:? Then
>> >>>>>>    existing clients that don't understand 312 might have a higher
>> >>>>>>    chance of at
>> >>>>>>    least doing something sensible.
>> >>>>>>
>> >>>>>>    If a client gets this response and starts downloading huge
>> content
>> >>>>>>    from the
>> >>>>>>    new IP and the client then opens a second connection to the
>> origin
>> >>>>>>    in a second
>> >>>>>>    tab. Which IPs is that supposed to use? The original ones or the
>> >>>>>>    redirected
>> >>>>>>    ones?
>> >>>>>>
>> >>>>>>    Requring user-agent snooping for a server to figure out if the
>> >>>>>>    feature works
>> >>>>>>    or not is a totally broken idea and I think this detail needs
>> to be
>> >>>>>>    worked out
>> >>>>>>    for this idea to be considered for real.
>> >>>>>>
>> >>>>>>    My personal preference is probably to add some sort of "urgency"
>> >>>>>>    thing to
>> >>>>>>    alt-svc instead of this 312 plus several headers, so that a
>> client
>> >>>>>>    can be told
>> >>>>>>    that it should switch sooner rather than later.
>> >>>>>>
>> >>>>>>    --
>> >>>>>>
>> >>>>>>       / daniel.haxx.se <http://daniel.haxx.se>
>> >>>>>>
>> >>>>>>
>> >>>>>>
>> >>>>>> --
>> >>>>>>
>> >>>>>> Bin Ni
>> >>>>>> VP of Engineering
>> >>>>>>
>> >>>>>> Quantil
>> >>>>>>
>> >>>>>> Connecting users with content...it's that simple.
>> >>>>>>
>> >>>>>> Office: +1-888-847-9851 <tel:(888)%20847-9851>
>> >>>>>>
>> >>>>>> Tweeter <https://twitter.com/Team_Quantil> Google Plus
>> >>>>>> <https://plus.google.com/+Quantil_team/> Linked In
>> >>>>>> <https://www.linkedin.com/company/quantil>
>> >>>>>>
>> >>>>>> The information contained in this email may be confidential and/or
>> >>>>>> legally privileged. It has been sent for the sole use of the
>> intended
>> >>>>>> recipient(s). If the reader of this message is not an intended
>> >>>>>> recipient, you are hereby notified that any unauthorized review,
>> use,
>> >>>>>> disclosure, dissemination, distribution, or copying of this
>> >>>>>> communication, or any of its contents, is strictly prohibited. If
>> you
>> >>>>>> have received this communication in error, please reply to the
>> sender
>> >>>>>> and destroy all copies of the message. To contact us directly,
>> send to
>> >>>>>> QUANTIL, INC. at 1919 S Bascom Ave #600, Campbell, CA 95008
>> >>>>>> <
>> https://maps.google.com/?q=1919+S+Bascom+Ave+%23600,+Campbell,+CA+95008&entry=gmail&source=g
>> >,
>> >>>>>> or visit our website at www.quantil.com. <https://www.quantil.com/
>> >
>> >>>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> --
>> >>>>>
>> >>>>> Bin Ni
>> >>>>> VP of Engineering
>> >>>>>
>> >>>>> Connecting users with content...it's that simple.
>> >>>>>
>> >>>>> Office: +1-888-847-9851
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>>
>> >>>>> The information contained in this email may be confidential and/or
>> legally privileged. It has been sent for the sole use of the intended
>> recipient(s). If the reader of this message is not an intended recipient,
>> you are hereby notified that any unauthorized review, use, disclosure,
>> dissemination, distribution, or copying of this communication, or any of
>> its contents, is strictly prohibited. If you have received this
>> communication in error, please reply to the sender and destroy all copies
>> of the message. To contact us directly, send to QUANTIL, INC. at 1919 S
>> Bascom Ave #600, Campbell, CA 95008, or visit our website at
>> www.quantil.com.
>> >>>>>
>> >>>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> --
>> >>>>
>> >>>> Bin Ni
>> >>>> VP of Engineering
>> >>>>
>> >>>> Connecting users with content...it's that simple.
>> >>>>
>> >>>> Office: +1-888-847-9851
>> >>>>
>> >>>>
>> >>>>
>> >>>>
>> >>>> The information contained in this email may be confidential and/or
>> legally privileged. It has been sent for the sole use of the intended
>> recipient(s). If the reader of this message is not an intended recipient,
>> you are hereby notified that any unauthorized review, use, disclosure,
>> dissemination, distribution, or copying of this communication, or any of
>> its contents, is strictly prohibited. If you have received this
>> communication in error, please reply to the sender and destroy all copies
>> of the message. To contact us directly, send to QUANTIL, INC. at 1919 S
>> Bascom Ave #600, Campbell, CA 95008, or visit our website at
>> www.quantil.com.
>> >>
>> >>
>> >>
>> >> --
>> >>
>> >> Bin Ni
>> >> VP of Engineering
>> >>
>> >> Connecting users with content...it's that simple.
>> >>
>> >> Office: +1-888-847-9851
>> >>
>> >>
>> >>
>> >>
>> >> The information contained in this email may be confidential and/or
>> legally privileged. It has been sent for the sole use of the intended
>> recipient(s). If the reader of this message is not an intended recipient,
>> you are hereby notified that any unauthorized review, use, disclosure,
>> dissemination, distribution, or copying of this communication, or any of
>> its contents, is strictly prohibited. If you have received this
>> communication in error, please reply to the sender and destroy all copies
>> of the message. To contact us directly, send to QUANTIL, INC. at 1919 S
>> Bascom Ave #600, Campbell, CA 95008, or visit our website at
>> www.quantil.com.
>> >
>> >
>>
>> --
>> Mark Nottingham   https://www.mnot.net/
>>
>>