Tcp window scaling is a key design in tcp protocol to improve network performance. Today we will dive into this topic to see how tcp window scale works in Linux.
- What is tcp window scaling?
- Benefit of tcp_window_scaling
- How to Check tcp_window_scaling in Linux?
- How to enable tcp_window_scaling?
- Tips for tcp_window_scaling
- Example of tcp_window_scaling
Table of Contents
What is tcp window scaling?
The TCP Window Scale option allows window size larger than 65K bytes by using a scale factor to multiply the window size value. This factor is set based on maximum size of receive buffer used by TCP sockets.
In the following example, maximum size of receive buffer is 4194304, so the scale factor is 7.
net.ipv4.tcp_rmem = 4096 87380 4194304
Note: The largest supported scaling factor is 14. This allows TCP window sizes of up to one Gigabyte.We can get more info from here. https://www.ietf.org/rfc/rfc1323.txt
Benefit of tcp_window_scaling
The base window size can not exceed 65535 bytes due to the limitations of the TCP header. This makes the window size too small.This option improves network bandwidth a lot. We can get the window size up to 1GB.Here’s the formula for calculating the maximum throughput of a single TCP connection: Window size / (RTT latency in milliseconds / 1,000) = maximum bytes/second
This table shows the maximum megabytes/per second throughput of a single TCP connection.
After tcp window scale is enabled, we can see that the bandwidth improves a lot.
Related:
How to Check tcp_window_scaling in Linux?
This is enabled by default in most Linux distributions. We can check it by the following command. cat /proc/sys/net/ipv4/tcp_window_scaling
If the output is 1, that means tcp window scaling is enabled.If the output is 0, that means tcp window scaling is disabled.
How to enable tcp_window_scaling?
We can use the following command to enable TCP window scaling. sysctl -w net.ipv4.tcp_window_scaling=1
To make this reboot persistent, we can add the following line to /etc/sysctl.conf. net.ipv4.tcp_window_scaling=1
Tips for Tcp_window_scaling
The window scaling option only appears in the first two packets of a TCP connection, it cannot be changed afterwards.
This only works by enabling this option on both sender and receiver ends. This wouldn’t improve the performance of existing connection since the tcp handshark was finished before. We need to restart the tcp connection to utilize this.
Window Scaling only works if both sides include it as an option during the initial 3-way handshake (even if their multipliers are different). The protocol recommends using the option even if the shift-count is 0 (meaning 2^0, which would multiply the window size by 1). That way, if one side doesn’t plan on using scaling, it doesn’t prevent the other side from taking advantage of the increased buffer space.
Example of tcp window scaling
The scp speed between our two US datacenters was about 1-2MB/s. From the network packets, it turns out that this option was not enabled on one server. We can get more info about how to use tcpdump to capture packets from here.
After tcp window scale is enabled, the scp speed reaches 40MB/s. This is a huge improvement. This post from Google also shares the same info.
Tcp window size in Wireshark
* Window size is the current size of the window in bytes.
* Scaling factor is a multiplier sent back along with the acknowledgment by the receiver that indicates to the sender that a new window size is requested.
* Calculated window size is the new window size that has been requested. This is determined like so: Window Size * Scaling factor = Calculated window size.
So, we would see on our wireshark output something like this:
window size value:593
{Calculated window size: 151808}
{Window size scaling factor: 256}
We can confirm the above by doing the math: 593*256 = 151808
Related post: