The transport layer (Layer 4) is where the two most important protocols in networking live: TCP and UDP. Every port scan you run, every reverse shell you catch, every denial-of-service you launch — they all depend on understanding these two protocols. This lesson gives you a deep, hacker-focused understanding of how they work and how they break.
TCP is a connection-oriented, reliable transport protocol. Before any data is exchanged, TCP establishes a connection through a process called the three-way handshake. It guarantees ordered delivery, retransmits lost packets, and manages flow control. Most critical services — web (HTTP/HTTPS), SSH, FTP, SMTP — use TCP because reliability matters.
The TCP three-way handshake is the foundation of every TCP connection. Understanding it is essential for port scanning, session hijacking, and firewall evasion.
TCP Three-Way Handshake:
Client Server
│ │
│──── SYN (seq=x) ───────>│ Step 1: Client initiates
│ │
│<── SYN-ACK (seq=y, ────│ Step 2: Server acknowledges
│ ack=x+1) │ and initiates
│ │
│──── ACK (ack=y+1) ────>│ Step 3: Client acknowledges
│ │ Connection ESTABLISHED
│<═══ Data Transfer ═════>│TCP headers contain 6 flag bits that control the connection. These flags are the building blocks of port scanning techniques and many network attacks.
| Flag | Purpose | Hacker Use Case |
|---|---|---|
| SYN | Synchronize — initiate connection | SYN scan (half-open scanning) |
| ACK | Acknowledge — confirm receipt | ACK scan (firewall detection, mapping rulesets) |
| FIN | Finish — gracefully close connection | FIN scan (bypass stateless firewalls) |
| RST | Reset — abort connection | Response to unexpected packets; used in RST attacks |
| PSH | Push — send data immediately | Forces data delivery; less commonly exploited |
| URG | Urgent — priority data | Rarely used; sometimes seen in IDS evasion |
UDP is connectionless and unreliable — it sends datagrams without establishing a connection, without guaranteeing delivery, and without ordering. This makes it fast but fragile. DNS queries, DHCP, TFTP, SNMP, and many real-time applications (VoIP, gaming) use UDP because speed matters more than reliability.
From a hacker's perspective, UDP's stateless nature makes it harder to scan (no handshake to observe) but also easier to abuse for amplification attacks, since UDP services can be tricked into sending large responses to small requests.
TCP vs UDP Comparison:
TCP: Connection-oriented UDP: Connectionless
Reliable Unreliable
Ordered delivery No ordering
Flow control No flow control
Slower Faster
3-way handshake No handshake
HTTP, SSH, FTP DNS, DHCP, SNMP, TFTPBoth TCP and UDP use 16-bit port numbers (0–65535) to identify services. Ports are divided into three ranges:
💡 When you see a reverse shell connecting back to your machine, it connects to a port you specify (commonly 4444). That port is the 'listener' on your attack box. The compromised machine uses an ephemeral port as its source.
TCP connections are terminated with a four-way FIN/ACK exchange. However, if one side sends an RST packet instead, the connection is immediately aborted. Attackers can inject RST packets to kill existing connections — a technique used in TCP session hijacking and connection reset attacks.
⚠️ TCP SYN flood attacks (sending massive SYN packets without completing the handshake) are illegal against systems you do not own. They are a classic DoS technique but carry severe legal penalties if used without authorization.
Verify exercises to earn ★ 150 XP and unlock next lab level.