Data Centers and TCP Incast – Georgia Tech – Network Congestion

Data Centers and TCP  Incast – Georgia Tech – Network Congestion


We’ll now talk about TCP congestion control in the context of modern datacenters. And we’ll talk about a particular TCP throughput collapse problem called the TCP incast problem. A typical data center consists of a set of server racks. Each holding a large number of servers, the switches that connect those racks of servers, and the connecting links that connect those switches to other parts of the topology. So the network architecture is typically made up of some sort of tree. And, switching elements that progressively are more specialized and expensive as we move up the network hierarchy. Some of the characteristics of a data center network include a high fan in, there is a very high amount of fan in between the leaves of the tree and the top of the root. Workloads are high bandwidth and low latency, and many clients issue requests in parallel, each with a relatively small amount of data per request. The other constraint that we face, is that the buffers in these switches can be quite small. So when we combine the requirements of high bandwidth and low latency for the applications, the presence of many parallel requests coming from these servers. And the fact that the switches have relatively small buffers, we can see that potentially there will be a problem. The throughput collapse that results from this phenomenon is called the TCP Incast problem. Incast is a drastic reduction in application throughput that results when servers using TCP. All simultaneously request data, leading to a gross underutilization of network capacity in many-to-one communication networks like a datacenter. The filling up of the buffers here at the switches result in bursty retransmissions that overfill the switch buffers. And these bursting retransmissions are cause by TCP timeouts. The TCP timeouts can lasts hundreds of milliseconds. But the roundtrip time in a data center network, is typically less than a millisecond. Often just hundreds of microseconds. Because the roundtrip times are so much less than TCP timeouts. The centers will have to wait for the TCP timeout. Before they retransmit. An application throughput can be reduced by as much as 90% as a result of link idle time.

Author: Kevin Mason

1 thought on “Data Centers and TCP Incast – Georgia Tech – Network Congestion

Leave a Reply

Your email address will not be published. Required fields are marked *