TCP is a seriously complicated protocol. It has to solve a long list of problems, and every solution seems to expose several smaller problems, edge cases, and dark corners. Learning TCP is not exactly a relaxing experience, but the process is rewarding: once you understand it, many things that used to feel vague or half-true suddenly become clear.
For the full technical treatment, W. Richard Stevens’ TCP/IP Illustrated, Volume 1: The Protocols is still worth reading. RFC 793 is the classic starting point too, followed by a rather large pile of later RFCs. I’ll use the common English terms throughout—Sequence Number, ACK, Window, SYN, and so on—because those are the keywords you will need when searching technical documentation.
The goal here is not to cover every corner of TCP. TCP has accumulated more than thirty years of optimizations, variants, arguments, and revisions. Trying to explain all of it in one sitting would be absurd. This part focuses on the protocol’s basic shape, its state transitions, and what happens when packets are lost. Flow control and congestion handling belong in the next part.
Before looking at TCP itself, place it in the network stack. In the OSI seven-layer model, TCP sits at layer 4, the Transport layer. IP is at layer 3, the Network layer. ARP is at layer 2, the Data Link layer. Data at layer 2 is usually called a Frame, data at layer 3 a Packet, and data at layer 4 a Segment.
Application data is first placed into TCP Segments. Those TCP Segments are then wrapped inside IP Packets. The IP Packets are then wrapped into Ethernet Frames and sent across the network. At the receiving end, each layer parses its own protocol headers and passes the payload upward to the next layer.
The TCP header
Here is the basic TCP header layout:
A few details matter immediately:
- A TCP segment does not contain IP addresses. IP addresses belong to the IP layer. TCP has source and destination ports.
- A TCP connection is identified by a tuple. In everyday discussion we usually say four-tuple:
(src_ip, src_port, dst_ip, dst_port). Strictly speaking, the protocol is also part of the identifier, making it a five-tuple. Since we are only discussing TCP here, the four-tuple is enough. - Four fields in the TCP header are especially important:
- Sequence Number identifies the byte position of the data in the stream. It is how TCP deals with packet reordering.
- Acknowledgement Number, or ACK, tells the peer what has been received. It is a key part of making the stream reliable.
- Window, also called Advertised Window, is the famous Sliding Window mechanism used for flow control.
- TCP Flags describe the type or purpose of a segment and drive TCP’s state machine.
Another diagram of the header fields:
TCP’s state machine
Strictly speaking, there is no such thing as a physical “connection” on the network, and TCP is no exception. What TCP calls a connection is really state maintained by both endpoints. The two sides remember enough information to make communication behave as if a connection exists.
That is why TCP’s state transitions are so important.
The following diagrams show TCP’s state machine together with the common connection setup, connection teardown, and data transfer paths. These diagrams are worth studying carefully. Once you see how much state is involved, you also get a sense of why TCP has so many traps.
A common question is: why does connection setup require a three-way handshake, while connection teardown is usually described as a four-way wave?
For the three-way handshake, the main purpose is to initialize the Sequence Number values on both sides. Each endpoint must tell the other its initial Sequence Number, usually abbreviated as ISN, for Initial Sequence Number. This is the reason for SYN: Synchronize Sequence Numbers. In the diagram those initial values are x and y. The numbers become the basis for ordering later data so that the application receives a clean byte stream even if packets arrive out of order on the network.
For the so-called four-way wave, if you look closely, it is really two shutdown operations. TCP is full-duplex, so each direction of the connection must be closed independently. Each side needs to send a FIN and receive an ACK. Because one side is usually passive, it appears as four separate steps. If both sides close at the same time, the connection may enter the CLOSING state and then move into TIME_WAIT.
Here is simultaneous close:
A few details around setup and teardown are worth calling out.
SYN timeout during connection setup
Imagine a server receives a SYN from a client and replies with SYN-ACK. Before the client sends the final ACK, it disappears. The server is now stuck in an intermediate state: the connection is neither established nor completely failed.
If the server does not receive the ACK within a certain time, TCP retransmits the SYN-ACK. On Linux, the default retry count is 5. The retry interval starts at 1 second and doubles each time: 1s, 2s, 4s, 8s, and 16s. That totals 31 seconds before the fifth retry has been sent. After that, the server still waits another 32 seconds to decide the fifth attempt timed out. In total:
1s + 2s + 4s + 8s + 16s + 32s = 2^6 - 1 = 63s
Only then does TCP give up on that connection attempt.
SYN Flood attacks
This behavior creates an opening for SYN Flood attacks. An attacker sends SYN packets to a server and then disappears. The server keeps half-open connection state around for the default timeout period, and enough of these requests can exhaust the SYN queue so legitimate connections cannot be handled.
Linux provides a parameter called tcp_syncookies to mitigate this. When the SYN queue is full, TCP can generate a special Sequence Number from the source address and port, destination address and port, and a timestamp. This value is sent back as a cookie. An attacker will not respond correctly. A real client will return the cookie, and the server can establish the connection even though the request was not kept in the SYN queue.
But do not treat tcp_syncookies as a normal load-handling tool. SYN cookies are a compromise version of TCP behavior and are not as strict as the normal protocol path. For legitimate high-load situations, consider tuning ordinary TCP parameters first:
tcp_synack_retriescan reduce the number of retries.tcp_max_syn_backlogcan increase the SYN queue size.tcp_abort_on_overflowcan simply reject connections when the server cannot keep up.
ISN initialization
The ISN must not be hard-coded. If every connection always starts with an ISN of 1, serious confusion can occur.
Suppose a client establishes a connection and sends 30 segments. The network breaks. The client reconnects, again using ISN 1. If old packets from the previous connection arrive late, the server may mistake them for packets belonging to the new connection. The client may think its Sequence Number is around 3, while the server believes it has already seen data up to around 30. Everything is now mixed up.
RFC 793 describes ISN generation as tied to a fake clock. The clock increments the ISN every 4 microseconds until it wraps past 2^32 and starts again from 0. That gives an ISN cycle of about 4.55 hours.
This assumes TCP Segments do not survive in the network longer than the Maximum Segment Lifetime, or MSL. As long as MSL is less than 4.55 hours, the same ISN should not be reused while old segments could still exist.
MSL and TIME_WAIT
The state diagram shows a timeout from TIME_WAIT to CLOSED. That timeout is 2 * MSL. RFC 793 defines MSL as 2 minutes. Linux uses 30 seconds.
Why have TIME_WAIT at all? Why not move straight to CLOSED?
There are two main reasons.
First, TIME_WAIT gives the peer enough time to receive the final ACK. If the passive closer does not receive that ACK, it may retransmit its FIN. One round trip of FIN and ACK can consume up to two MSLs.
Second, TIME_WAIT helps prevent packets from an old connection being confused with packets from a later connection using the same tuple. Some routers may cache IP packets or delay them unexpectedly. If the same connection identifiers are reused too quickly, delayed packets from the old connection may appear inside the new one.
Too many TIME_WAIT sockets
TIME_WAIT is important, but under high-concurrency short-lived connections it can become numerous and consume system resources. A quick search will often turn up suggestions to enable two Linux parameters: tcp_tw_reuse and tcp_tw_recycle. Both are disabled by default. tcp_tw_recycle is more aggressive; tcp_tw_reuse is somewhat gentler. If tcp_tw_reuse is used, tcp_timestamps=1 must also be enabled, otherwise it has no effect.
Be very careful. Enabling these parameters can cause strange TCP connection failures. If connections are reused before the protocol’s timeout rules have done their job, new connections may fail in surprising ways. Official documentation warns that these settings should not be changed without advice or request from technical experts.
For tcp_tw_reuse, documentation says that with tcp_timestamps—also known as PAWS, Protection Against Wrapped Sequence Numbers—the behavior can be safe from a protocol perspective. But timestamps must be enabled on both sides. Even then, some scenarios may still be problematic.
For tcp_tw_recycle, things are riskier. If it is enabled, the kernel assumes the peer uses TCP timestamps and compares timestamp values. If the timestamp has increased, the connection can be reused. But when the peer is behind NAT—say a company using one public IP for many machines—or when an IP address gets reused by another host, the timestamp assumptions break down. SYN packets may be dropped outright, producing connection timeouts.
There is also tcp_max_tw_buckets, which controls the number of concurrent TIME_WAIT sockets. The default is 180000. If the limit is exceeded, the system destroys extra entries and logs warnings such as time wait bucket table overflow. Official documentation describes this parameter as a defense against DDoS-style pressure, and the default value is not small. It should be adjusted according to actual conditions, not blindly.
Again: using tcp_tw_reuse or tcp_tw_recycle as a shortcut for solving TIME_WAIT pressure is dangerous, because these behaviors violate the expectations of TCP as specified in RFC 1122.
Also remember what TIME_WAIT means: it is the side that actively closes the connection that enters it. If the peer closes first, the problem belongs to the peer. For an HTTP server, this is one reason HTTP KeepAlive matters. A browser can reuse a single TCP connection for multiple HTTP requests. Then, in many cases, the client side can be the one to close. Of course, browsers may also be greedy and keep connections open until they really have to close them.
Sequence Number during data transfer
Here is a Wireshark flow graph showing data transfer during a web request. It illustrates how SeqNum changes over time.
The important point: SeqNum increases according to the number of bytes transmitted.
In the screenshot, after the three-way handshake, two packets arrive with Len:1440. The second packet’s SeqNum becomes 1441. Then the ACK for the first packet is 1441, meaning the first 1440 bytes were received.
One small Wireshark trap: if you capture a three-way handshake, you may see SeqNum displayed as 0. That does not mean the real Sequence Number is 0. Wireshark uses Relative SeqNum by default for readability. Disable that option in protocol preferences if you want to see the absolute Sequence Number.
TCP retransmission
TCP promises reliable delivery, so it needs retransmission.
The receiver’s ACK confirms only the last continuous byte range it has received. Suppose the sender transmits five chunks: 1, 2, 3, 4, and 5. The receiver gets 1 and 2, so it sends ACK 3. Then it receives 4, but 3 has not arrived.
What should TCP do?
Because Sequence Number and ACK are byte-based, ACK cannot skip over a missing section. The receiver cannot say “I got 4” using the ordinary ACK field while 3 is missing. It can only acknowledge the largest continuous range. Otherwise the sender would assume everything before that ACK had arrived.
Timeout retransmission
One approach is to send no new ACK beyond the missing point and wait for 3. When the sender times out waiting for an ACK that covers 3, it retransmits 3. Once the receiver gets 3, it can ACK 4, which means both 3 and the already-received 4 are now covered.
This has a serious drawback: everything waits on packet 3. Even if 4 and 5 have arrived, the sender does not really know that from ordinary ACKs. Since no advancing ACK is received, the sender may pessimistically assume that 4 and 5 were also lost, leading to unnecessary retransmissions.
There are two common choices here:
- Retransmit only the timed-out packet, such as data chunk 3.
- Retransmit the timed-out packet and everything after it, such as 3, 4, and 5.
Both choices are imperfect. The first saves bandwidth but may be slower. The second can recover faster but wastes bandwidth and may redo work unnecessarily. The deeper issue is that both wait for a timeout, and a timeout can be long. TCP’s dynamic timeout calculation is a topic of its own.
Fast Retransmit
TCP therefore introduced Fast Retransmit. Instead of being driven purely by time, retransmission can be driven by incoming data patterns.
If packets do not arrive continuously, the receiver keeps acknowledging the last missing point. When the sender receives three duplicate ACKs, it retransmits immediately instead of waiting for timeout.
Example: the sender transmits 1, 2, 3, 4, and 5. Packet 1 arrives, so the receiver sends ACK 2. Packet 2 is lost. Packet 3 arrives, but the receiver still sends ACK 2. Packets 4 and 5 arrive, and the receiver still sends ACK 2, because 2 is still missing. Once the sender sees three ACKs for 2, it knows 2 likely did not arrive and retransmits it. After the receiver gets 2, it already has 3, 4, and 5, so it can ACK 6.
Fast Retransmit solves the timeout waiting problem, but it still leaves a difficult question: should the sender retransmit only the missing segment or all later data as well?
In the example, should it retransmit only #2, or #2 through #5? The sender does not necessarily know which packets generated the three duplicate ACKs. If it sent 20 chunks, the duplicate ACKs might have come from #6, #10, and #20. Some TCP implementations may retransmit a large range starting at the missing point. Fast Retransmit is useful, but it is still a double-edged sword.
SACK: Selective Acknowledgment
A better method is Selective Acknowledgment, or SACK, described in RFC 2018. SACK adds information to the TCP header so the receiver can report which blocks of data have arrived. The ordinary ACK still behaves like Fast Retransmit’s ACK, but SACK tells the sender about received fragments beyond the missing point.
With SACK, the sender can tell which data arrived and which data is missing, making Fast Retransmit much more precise. Both endpoints must support the option. On Linux, the tcp_sack parameter enables it, and it has been enabled by default since Linux 2.4.
There is one subtle issue: receiver reneging. Reneging means the receiver is allowed to discard data it previously reported in SACK. This is discouraged because it complicates the protocol, but it can happen in extreme situations, such as when the receiver must free memory for something more important.
Because of this, the sender cannot rely completely on SACK. It still must rely on cumulative ACK progress and maintain timeouts. If later ACKs do not advance, data previously reported by SACK may still need to be retransmitted. Likewise, the receiver must never treat SACKed data as fully ACKed in the cumulative sense.
SACK also consumes sender resources. If an attacker sends many SACK options to a sender, the sender may have to retransmit or traverse its outstanding data structures, consuming CPU and memory. SACK improves performance, but it has tradeoffs.
D-SACK: detecting duplicate data
Duplicate SACK, or D-SACK, uses SACK to tell the sender which data was received more than once. RFC 2883 describes the details and examples.
D-SACK uses the first SACK block as a marker:
- If the range in the first SACK block is covered by the cumulative ACK, it is D-SACK.
- If the range in the first SACK block is covered by the second SACK block, it is also D-SACK.
Example 1: ACK loss
In the following example, two ACKs are lost. The sender retransmits the first segment, 3000-3499. The receiver sees duplicate data and replies with SACK=3000-3500. Since ACK has already reached 4000, meaning all data before 4000 has been received, this SACK block is D-SACK. It tells the sender that the data packet was not lost; the ACKs were.
Transmitted Received ACK Sent
Segment Segment (Including SACK Blocks)
3000-3499 3000-3499 3500 (ACK dropped)
3500-3999 3500-3999 4000 (ACK dropped)
3000-3499 3000-3499 4000, SACK=3000-3500
---------
Example 2: network delay
In this example, the packet 1000-1499 is delayed by the network. The sender does not receive the expected ACK, and later packets trigger Fast Retransmit. But when the retransmission occurs, the delayed original packet also arrives. The receiver sends SACK=1000-1500. Since ACK has reached 3000, this is D-SACK, indicating duplicate reception.
Here, the sender can infer that retransmission was not caused by a lost data packet or a lost ACK. It was caused by network delay.
Transmitted Received ACK Sent
Segment Segment (Including SACK Blocks)
500-999 500-999 1000
1000-1499 (delayed)
1500-1999 1500-1999 1000, SACK=1500-2000
2000-2499 2000-2499 1000, SACK=1500-2500
2500-2999 2500-2999 1000, SACK=1500-3000
1000-1499 1000-1499 3000
1000-1499 3000, SACK=1000-1500
---------
D-SACK gives TCP several useful signals:
- It helps the sender distinguish between lost data packets and lost ACK packets.
- It can reveal whether the sender’s timeout is too small and causing unnecessary retransmission.
- It can show packet reordering, where earlier packets arrive after later ones.
- It can expose packet duplication somewhere in the network.
Those signals help TCP understand current network behavior and make better flow-control decisions.
On Linux, the tcp_dsack parameter enables D-SACK, and it has been enabled by default since Linux 2.4.