Re: UDP source ports for HTTP/3 and QUIC

On Fri, Jul 16, 2021 at 06:34:32AM +0000, Poul-Henning Kamp wrote:
> --------
> Willy Tarreau writes:
> 
> > Stefan made a good point about the problem that might result, with
> > inbound load balancing between multiple listeners (typically what's
> > achieved by L3 switches doing L3+L4 hash between multiple servers,
> > and operating systems hashing the source+destination port to pick a
> > different listening socket). Thus a suggestion might be to possibly
> > save resources by using a small amount of sockets, with "small" left
> > to the appreciation of the implementation.
> 
> We should run the question "few or many UDP ports?" past some some
> friendly 100G and 400G device driver maintainers.
> 
> The NICs I have been working with for the ESO ELT project all
> included UDP ports in the hash they used to decide which CPU core
> to deliver packets/interrupts to and we had to spread across UDP
> ports to or all the traffic would hit one single core.
> 
> I dont know if QUIC has registered with the NIC designers and device
> drivers writers yet, but given the opacity of QUIC packets, it is
> very hard to see what else than the UDP port they can feed into
> their hash, so I expect the answer to be "many".

That's true but queues are not the only parameter, as using many
sockets also results in a massive cache miss ratio and many more
syscalls. Even with TCP it's quite visible that using a few hundred
connections tends to be incredibly more efficient at 100Gbps than
using tens of thousands.

Ideally we want all queues to be used with a good balance while
limiting the amount of operations to transport these data.

Probably that some wording evocating that possibility offered by
the protocol and a suggestion to take various factors into account
(L4 LB, NIC queues, listeners, syscalls etc) is sufficient to let
implementors ask the relevant people for advice in their context.

Willy

Received on Friday, 16 July 2021 07:17:18 UTC