Tcp window scaling is a key design in tcp protocol to improve network performance. Today we will dive into this topic to see how tcp window scale works in Linux.
What is tcp window scaling?
The TCP Window Scale option allows window size larger than 65K bytes by using a scale factor to multiply the window size value. This factor is set based on maximum size of receive buffer used by TCP sockets.
In the following example, maximum size of receive buffer is 4194304, so the scale factor is 7.
net.ipv4.tcp_rmem = 4096 87380 4194304
Note: The largest supported scaling factor is 14. This allows TCP window sizes
of up to one Gigabyte.
We can get more info from here. https://www.ietf.org/rfc/rfc1323.txt
Benefit of tcp_window_scaling
The base window size can not exceed 65535 bytes due to the limitations of the TCP header. This makes the window size too small.
This option improves network bandwidth a lot. We can get the window size up to 1GB.
Here's the formula for calculating the maximum throughput of a single TCP connection:
Window size / (RTT latency in milliseconds / 1,000) = maximum bytes/second
This table shows the maximum megabytes/per second throughput of a single TCP connection.
After tcp window scale is enabled, we can see that the bandwidth improves a lot.
How to Check tcp_window_scaling in Linux?
This is enabled by default in most Linux distributions. We can check it by the following command.
If the output is 1, that means tcp window scaling is enabled.
If the output is 0, that means tcp window scaling is disabled.
How to enable tcp_window_scaling?
We can use the following command to enable TCP window scaling.
sysctl -w net.ipv4.tcp_window_scaling=1
To make this reboot persistent, we can add the following line to /etc/sysctl.conf
Tips for Tcp_window_scaling
The window scaling option only appears in the first two packets of a TCP connection, it cannot be changed afterwards.
This only works by enabling this option on both sender and receiver ends. This wouldn't improve the performance of existing connection since the tcp handshark was finished before. We need to restart the tcp connection to utilize this.
Example of tcp window scaling
The scp speed between our two US datacenters was about 1-2MB/s. From the network packets, it turns out that this option was not enabled on one server. We can get more info about how to use tcpdump to capture packets from here.
After tcp window scale is enabled, the scp speed reaches 40MB/s. This is a huge improvement. This post from Google also shares the same info.