0% completed
TCP Connections and Ports
On This Page
The Three-Way Handshake
Ports and Listening Sockets
Refused and Timed Out
Connection States and TIME_WAIT
Ephemeral Port Exhaustion
The Listen Backlog
Keepalive
UDP in Contrast
Using This in an Interview
Key Takeaways
Practice Questions
A health check calls a new service on port 8080 and fails with Connection refused. A second check, from another subnet (a separate range of network addresses), waits several seconds and fails with Connection timed out. Do the two errors point to the same problem?
They do not. A refused connection means that a packet reached the target host, which replied that nothing accepts connections on that port. A timed out connection means that no reply arrived at all, and this lesson explains why that difference tells you where to look next.
The Three-Way Handshake
TCP opens a connection with three packets before any application data moves. TCP (Transmission Control Protocol) delivers a stream of bytes in order and resends any that are lost. The client sends a SYN (synchronize) packet, the server replies with a SYN-ACK (synchronize and acknowledge), and the client answers with an ACK.
The handshake costs one round trip, the time for a packet to reach the other host and for its reply to return. If the round trip takes 40 ms, the client can send its first request about 40 ms after it starts.
Ports and Listening Sockets
An IP address selects a host, and a port number from 0 to 65535 selects one program on that host. A socket is the endpoint through which a program sends and receives network data. A listening socket waits for new connections on one address and port, and each accepted connection receives its own socket.
The kernel identifies each connection by four values: source address, source port, destination address, and destination port. To list the listening sockets, run ss with four flags: -t for TCP, -l for listening, -n for numeric output, and -p for the owning process.
sudo ss -tlnp
State Recv-Q Send-Q Local Address:Port Peer Address:Port Process LISTEN 0 4096 127.0.0.1:8080 0.0.0.0:* users:(("orders-api",pid=2211,fd=7)) LISTEN 0 511 0.0.0.0:443 0.0.0.0:* users:(("nginx",pid=980,fd=6))
The service listens on port 8080, but only on 127.0.0.1, the loopback address that accepts connections from the same host. A health check from another host is therefore refused, and the fix is to bind the service to 0.0.0.0 or to the host's network address.
Refused and Timed Out
A refused connection is a fast, explicit answer, while a timed out connection is the absence of any answer. When a SYN reaches a host and no socket listens on that port, the kernel replies with a reset (RST), a packet that ends the attempt. The client reports the error within one round trip.
When the SYN is lost or discarded, nothing returns. A firewall is a set of rules that allows or blocks traffic, and one configured to drop packets is the most common cause. The Linux client resends the SYN with growing waits, and with default settings it gives up after roughly two minutes. Most programs set a shorter connect timeout.
nc: connect to 10.0.3.15 port 8080 (tcp) failed: Connection refused nc: connect to 10.0.7.40 port 8080 (tcp) failed: Connection timed out
| Refused | Timed out | |
|---|---|---|
| What returned | A reset | Nothing |
| Time to fail | About one round trip | The client's connect timeout |
| What it proves | The network path to the host works | Something on the path discards packets, or the host is down |
| Common causes | Process not running, wrong port, bound to loopback | Firewall or security group (a cloud firewall), wrong route, host down |
| Next check | ss -tlnp on the target | Firewall rules and routes |
A refused connection usually rules out the network path. However, a firewall configured to reject rather than drop also produces a refusal, so check the rules when the process is clearly listening.
Connection States and TIME_WAIT
Each socket moves through named states, and ss shows the current state of every connection. ESTABLISHED means data can flow. CLOSE-WAIT means the other side has closed, but the local program has not yet closed its socket, so a growing count usually indicates a bug in that program.
The side that closes first enters TIME_WAIT for 60 seconds on Linux, so that late packets from the old connection are not accepted by a new connection with the same four values.
Ephemeral Port Exhaustion
A client takes its source port from the ephemeral range, and connections to one destination share that range. An ephemeral port is a temporary port that the kernel assigns to an outgoing connection. The Linux default range is 32768 to 60999, which is 28,232 ports.
Consider a service that opens a new connection for every query to a database proxy at 10.0.5.20, port 5432, and closes it first. Each source port then stays in TIME_WAIT for 60 seconds. The service can therefore open about 28,232 / 60, or roughly 470, new connections each second to that destination. At 600 queries a second, new connections fail with Cannot assign requested address.
The best fix is a connection pool, a set of open connections that requests share. Widening the port range only raises the limit.
The Listen Backlog
The kernel completes the handshake on its own and queues the connection until the program accepts it. That queue is the listen backlog. Its size comes from the program's listen() call, capped by the net.core.somaxconn setting.
For a listening socket, ss reports the current queue length in Recv-Q and the maximum in Send-Q. In the output above, nginx's queue has room for 511 connections and is empty. When a slow program lets the queue fill, Linux by default drops new handshakes instead of resetting them. Clients therefore see slow connections or timeouts, although the port is open, and nstat -az TcpExtListenOverflows counts the drops.
Keepalive
TCP keepalive sends small probes on an idle connection to confirm that the other side still exists. It is off until the program enables it on the socket. Linux then waits 7,200 seconds of idle time before the first probe, and then sends up to 9 probes 75 seconds apart.
Many load balancers and NAT gateways delete idle connections after a few minutes. A NAT gateway lets many private hosts share one public address. When such a device deletes an idle pooled connection, the next request on it receives a reset or no reply. Set the keepalive time, or the pool's maximum idle time, below the device's idle timeout. HTTP keep-alive is a different feature, in which one connection carries many requests.
UDP in Contrast
UDP (User Datagram Protocol) sends independent messages with no handshake, no connection state, and no resending, which suits small DNS queries. A message to a closed UDP port may cause a "port unreachable" reply over ICMP, the protocol that carries network error messages. Firewalls often drop that reply, so a UDP test that reports no error proves little.
Using This in an Interview
A candidate report from an SRE loop lists TCP versus UDP among the networking questions. Connect each answer to how failures look.
"Refused means the SYN reached the host and the kernel sent a reset because nothing listens on that port, so I check the process with
ss -tlnp, including whether it is bound only to loopback. Timed out means no reply, so I check firewall rules, security groups, and routes. A full listen backlog can also cause timeouts on an open port."
Expect two follow-up questions:
- "A host has 30,000 sockets in TIME_WAIT. Is that a problem?" Only if it exhausts ephemeral ports to one destination. Check for connect errors, and reuse connections.
- "Why does CLOSE-WAIT keep growing?" The peer closed its side, but the local program never called close, so its unused sockets stay open.
Key Takeaways
- Refused proves that the host is reachable; timed out points to a firewall, a route, or a host that is down.
ss -tlnpshows what listens, on which address, and in which process.- TIME_WAIT and the ephemeral port range limit new connections to one destination, so reuse connections.
- A full listen backlog produces timeouts, not refusals.
A connection that opens correctly can still fail when encryption begins. The next lesson, TLS and Certificates, explains the handshake that follows the TCP handshake and the certificate errors it produces.
Practice Questions
1. A port test to a server fails immediately with "Connection refused." Is a firewall that drops packets the likely cause?
<details><summary>Show answer</summary>No, because a dropping firewall produces a timeout, not an immediate refusal. The reset shows that the SYN reached the host. Check whether the process runs, which port it uses, and whether it listens only on 127.0.0.1.
</details>2. A client opens and closes a new connection for each request to one server, closing first. Roughly how many new connections per second can it open with the default port range?
<details><summary>Show answer</summary>About 470 per second. The range 32768 to 60999 contains 28,232 ports, and each port stays in TIME_WAIT for 60 seconds. Dividing 28,232 by 60 gives about 470.
</details>3. ss -tlnp shows a listening socket with Recv-Q 129 and Send-Q 128. What does that mean, and what will clients see?
The listen backlog is full, so new handshakes are dropped. Linux admits one connection beyond the backlog, so this queue is full. The program is not accepting connections as fast as they arrive. Clients see slow connections or timeouts even though the port is open.
</details>Reading Progress
0%
On This Page
The Three-Way Handshake
Ports and Listening Sockets
Refused and Timed Out
Connection States and TIME_WAIT
Ephemeral Port Exhaustion
The Listen Backlog
Keepalive
UDP in Contrast
Using This in an Interview
Key Takeaways
Practice Questions