Johnathan's Blog

This is my blog. There are many like it, but this one is mine.

ss explained

- (20 min read)

I have typed ss thousands of times. For most of those, I was really only asking it one question: "what is listening on this box, and who is talking to it?" I would run ss -tlnp, skim the output, and move on. The columns to the right of the addresses were noise I had trained myself to ignore.

That is a poor way to know a tool. So this is me teaching it to myself, out loud, one column and one flag at a time. If you already live in ss, you can skip to the state machine or the filtering sections, which are the parts most people (me included) underuse.

A quick note on what ss even is. It ships in iproute2, the same package that gives you ip. It reads socket information straight from the kernel over a netlink socket (sock_diag), which is why it is faster than netstat. netstat walks /proc/net/tcp and friends, parsing text files line by line; ss asks the kernel directly. The name stands for "socket statistics". It is the tool you should reach for now that netstat is deprecated on most distributions.

Installing ss

You almost certainly already have it. iproute2 is a base package on essentially every modern Linux system, so ss is present out of the box on anything you are likely to SSH into. If it is somehow missing:

# Debian, Ubuntu
apt install iproute2

# RHEL, CentOS, Fedora, Rocky, Alma
dnf install iproute

# Alpine
apk add iproute2

# Arch
pacman -S iproute2

Confirm the version, because the output format and some flags have shifted over the years:

$ ss --version
ss utility, iproute2-6.1.0

Everything below reflects a reasonably recent iproute2 (5.x and up). If you are on something ancient, the columns are the same but a few of the richer flags (--cgroup, --tos) may be absent.

Running it bare

Start with no arguments at all. This is where most people never look, because the output is a wall of sockets you did not ask about.

$ ss
Netid State  Recv-Q Send-Q  Local Address:Port          Peer Address:Port
u_str ESTAB 0      0       /run/systemd/journal/stdout 24807               * 24806
u_str ESTAB 0      0       * 0                          * 0
tcp   ESTAB 0      0       192.168.1.50:ssh            192.168.1.10:52814
tcp   ESTAB 0      0       192.168.1.50:47320          140.82.113.25:https

Two things surprise people here. First, bare ss shows every socket family it knows about, not just TCP. Those u_str rows are Unix domain sockets, the local IPC channels that daemons use to talk to each other. Second, it shows only connected sockets by default. Nothing that is merely listening appears until you ask for it with -l or -a. So the default view answers "who is currently talking", which is a reasonable default, and also the opposite of what I usually want.

Let me walk the columns, because every other invocation is just a variation on these.

The columns

Netid

The socket family. You will mostly see:

  • tcp, udp for the internet transports you expect.
  • u_str, u_dgr, u_seq for Unix domain sockets (stream, datagram, seqpacket). These are the local sockets under paths like /run/....
  • nl for netlink, the kernel's own control-plane sockets.
  • raw for raw IP sockets (think ping).
  • p_raw, p_dgr for packet sockets (think tcpdump).

When you constrain the output to a single family with a flag like -t, this column disappears, since it would say the same thing on every row.

State

The lifecycle stage of the socket. For TCP this is a rich set drawn straight from the TCP state machine, and it is worth its own section below. For the connectionless families it collapses to two useful values: ESTAB when a socket has a peer wired up, and UNCONN when it does not. A bound UDP socket that has not connect()ed to a specific peer sits in UNCONN, which reads oddly the first time you see it, since UDP has no connections in the TCP sense.

Recv-Q and Send-Q

Here is the field I ignored for years, and the one with a genuine trap in it: these two columns mean completely different things depending on whether the socket is listening or connected.

For a connected socket:

  • Recv-Q is the number of bytes sitting in the kernel's receive buffer that your application has not read() yet. A number that climbs and stays high means your program is not draining its socket fast enough. The data arrived; nobody picked it up.
  • Send-Q is the number of bytes sitting in the send buffer that the kernel has handed to the network but the peer has not acknowledged. A persistently high Send-Q points the finger the other way: the remote end, or the path to it, is not keeping up.

For a listening socket the same two columns are repurposed entirely:

  • Recv-Q is the number of connections currently sitting in the accept queue, fully established by the kernel and waiting for your application to call accept().
  • Send-Q is the maximum size of that accept queue, which is the backlog you passed to listen(), capped by net.core.somaxconn.

This second meaning is the useful one and almost nobody knows it. If you are debugging a service that drops connections under load, look at a listening socket's Recv-Q against its Send-Q:

$ ss -tln
State  Recv-Q Send-Q Local Address:Port Peer Address:Port
LISTEN 0      511    0.0.0.0:80         0.0.0.0:*
LISTEN 0      128    0.0.0.0:22         0.0.0.0:*

Send-Q of 511 on port 80 is nginx's default backlog. If under load you saw Recv-Q creeping toward 511 on that row, your application is accepting connections slower than they arrive, and the kernel is about to start refusing them. That is a real diagnosis you can make at a glance, and it is invisible if you read these columns the way you read them on a connected socket.

Local Address:Port and Peer Address:Port

The two endpoints. A few conventions are worth internalizing:

  • * is the wildcard. *:* as a peer means "anything", which is what you see on listening sockets, since they have no single peer yet. 0.0.0.0 and [::] are the wildcard addresses for "all interfaces" on IPv4 and IPv6 respectively.
  • By default ss resolves both the address (to a hostname) and the port (to a service name from /etc/services). That is why you see ssh and https instead of 22 and 443 above. It is friendly and it is slow, since resolution can block on DNS. The -n flag turns it off, and you will want it off more often than on.
  • For IPv6 the address is bracketed, so a socket looks like [::1]:443. The port always follows the last colon, which is exactly why the brackets exist.
  • For Unix sockets the "address" is a filesystem path (or an abstract name), and the trailing number after it is the socket's inode. That is how you correlate the two ends of a Unix socket pair: matching inodes.

That is the entire default output. Everything else ss does is either filtering these rows down or hanging more detail off of them.

Socket states

TCP is a state machine, and ss shows you which state each connection is in. This is the ss equivalent of learning what R, S, D, and Z mean in a process list: once you know the states, the output starts telling you stories instead of just facts.

Here are the states you will actually encounter, roughly in the order a connection lives through them:

  • LISTEN A server socket waiting for incoming connections. It has a local address and port but no peer. This is what you are looking for when you ask "what is running on this machine".
  • SYN-SENT You initiated a connection (connect()) and sent the opening SYN, but the other side has not answered yet. A pile of these means you are trying to reach something that is not responding: a down host, a dropped packet, a firewall silently eating your SYNs.
  • SYN-RECV The mirror image. You received a SYN, replied with SYN-ACK, and are waiting for the final ACK. A flood of SYN-RECV from many different peers is the classic signature of a SYN flood, or of clients disappearing mid-handshake.
  • ESTAB The connection is open and data can flow both ways. The steady state, and the bulk of a healthy server's rows.
  • FIN-WAIT-1 and FIN-WAIT-2 You called close() and sent a FIN; now you are waiting for the peer to acknowledge it (FIN-WAIT-1) and then to send its own FIN (FIN-WAIT-2). Connections stuck in FIN-WAIT-2 usually mean the remote application closed its side lazily, or not at all.
  • CLOSE-WAIT The peer closed first, the kernel acknowledged it, and now the kernel is waiting for your application to call close(). This one is a smoking gun. A growing count of CLOSE-WAIT sockets almost always means a bug in your program: it is leaking connections, forgetting to close file descriptors after the other end hung up. The kernel cannot clean these up for you; only your close() can.
  • LAST-ACK You sent your FIN after being in CLOSE-WAIT and are waiting for the final acknowledgement. Transient.
  • TIME-WAIT The connection is closed, but the socket lingers for a while (typically 2 minutes, twice the maximum segment lifetime) to absorb any stray late packets and to make sure the peer received the final ACK. Thousands of TIME-WAIT sockets on a busy server are normal and usually harmless; they are the residue of connections that closed cleanly. They only become a problem when you exhaust ephemeral ports.
  • CLOSING Both sides sent FINs at nearly the same moment. Rare, and transient.
  • UNCONN Not a TCP state at all. It is what ss prints for a bound-but-not-connected socket, which you see on UDP and on listening sockets of families that do not have a LISTEN concept.

The practical payoff: the distribution of states is a health readout. Mostly ESTAB with a scattering of TIME-WAIT is a happy machine. A wall of CLOSE-WAIT is a leaking application. A wall of SYN-SENT is a reachability problem. A wall of SYN-RECV is something attacking your handshake. You can produce that distribution in one line, which I will get to under summary statistics.

Choosing what to show

Now the flags. The first group narrows ss down to the family and lifecycle you care about, and you will combine two or three of them almost every time.

Protocol selectors:

ss -t   # TCP only
ss -u   # UDP only
ss -x   # Unix domain sockets only
ss -w   # raw sockets
ss -4   # IPv4 only
ss -6   # IPv6 only

These stack. ss -tu shows TCP and UDP together; ss -t4 shows IPv4 TCP.

Lifecycle selectors:

ss -l   # listening sockets only
ss -a   # all sockets (listening and connected)

Remember that bare ss shows only connected sockets. -l flips that to only listening, and -a shows both. So the two invocations I actually use constantly are:

ss -tlnp    # what is listening (TCP), numeric, with the owning process
ss -tnp     # who is connected right now (TCP), numeric, with the process

That is the canonical "audit this box" command and the canonical "watch live traffic" command, respectively. -n for numeric is doing a lot of quiet work in both: it stops ss from blocking on reverse DNS for every foreign address, which on a server talking to the wider internet is the difference between instant and interminable.

Making it readable

A few flags exist purely to shape the output for eyes or for scripts.

  • -n numeric. Skip name resolution for both addresses and ports. Faster, and unambiguous.
  • -r resolve. The opposite: force resolution of numeric addresses back to names. The default already resolves, so you reach for this mostly when combined with other tools.
  • -H no header. Drop the header row. Essential when you are piping into awk or wc and do not want to special-case the first line.
  • -O oneline. Keep each socket's full record on a single line. The detail flags like -i normally wrap continuation data onto an indented second line; -O un-wraps it so each socket is one grep-able record.
  • -Q no queues. Suppress the Recv-Q and Send-Q columns when you do not care about them and want the addresses to line up more tidily.

For example, "count how many established connections there are" becomes clean with -H:

$ ss -tnH state established | wc -l
47

No header to subtract, no service-name resolution to wait on.

Who owns the socket

A socket without its owning process is half a fact. The -p flag closes that gap, and a couple of siblings add identity detail.

$ ss -tlnp
State  Recv-Q Send-Q Local Address:Port Peer Address:Port Process
LISTEN 0      128    0.0.0.0:22         0.0.0.0:*         users:(("sshd",pid=812,fd=3))
LISTEN 0      511    0.0.0.0:80         0.0.0.0:*         users:(("nginx",pid=1123,fd=6),("nginx",pid=1122,fd=6))

The Process column reads users:(("name",pid=N,fd=M)), and it can list several holders when a socket is shared across forked workers, as nginx does above. fd is the file descriptor number inside that process, which is occasionally handy when you go spelunking in /proc/PID/fd.

One caveat that trips everyone up once: -p only shows processes you have permission to see. Sockets owned by other users, including most system daemons, show up with an empty Process column unless you run ss as root. If your process column is mysteriously blank, prefix with sudo.

Two related flags add identity:

  • -e extended. Appends the owning uid, the socket inode, and a sk cookie that uniquely identifies the socket to the kernel. Useful for correlating a socket across tools or across successive ss runs.
  • -Z and -z context. On SELinux systems, show the security context of the process (-Z) or of the socket itself (-z).

Filtering by state

Everything so far narrows by family and by process. The more powerful axis is filtering by TCP state, and ss has a compact language for it. You write state NAME after the flags:

ss -tn state established
ss -tn state time-wait
ss -tn state close-wait

The state names are exactly the ones from the state machine above, lowercased and hyphenated: established, syn-sent, syn-recv, fin-wait-1, fin-wait-2, time-wait, closed, close-wait, last-ack, listening, closing.

What elevates this from convenient to genuinely useful are the aggregate aliases, which let you ask about whole categories of state at once:

  • connected every state except listening and closed. "Show me sockets that have a peer."
  • synchronized the connected states except syn-sent. "Show me sockets past the handshake."
  • bucket the minisocket states, time-wait and syn-recv, which the kernel keeps in a compact table rather than as full sockets.
  • big the inverse of bucket, meaning the full-socket states.
  • all every state, equivalent to -a.

So ss -tn state connected is a precise way to say "real conversations only, no listeners", and ss -tn state bucket isolates the lightweight leftovers that a busy server accumulates.

You can also negate. ss -tn exclude established gives you everything that is not in the steady state, which is a fast way to spot connections that are stuck mid-lifecycle.

Filtering by address and port

The second filter language matches on the endpoints. The predicates are src and dst for addresses, sport and dport for ports, and they compose with boolean operators.

The simplest form matches a port:

ss -tn dport = :443      # connections whose remote port is 443
ss -tn sport = :22       # connections whose local port is 22

Note the leading colon on :443. The general host syntax is ADDRESS:PORT, so a bare :443 means "any address, port 443". You can pin the address too:

ss -tn dst 140.82.113.25
ss -tn dst 140.82.113.25:443
ss -tn src 192.168.1.0/24        # CIDR ranges work
ss -tn 'dst != 127.0.0.0/8'      # everything not loopback

Ports understand comparison operators, not only equality, which is how you filter port ranges. The operators are <, <=, =, >=, >, and !=, with word aliases (eq, ne, gt, lt, ge, le) if you prefer them or if your shell is fighting you over the symbols:

ss -tn 'sport >= :1024'          # local ephemeral ports
ss -tn 'dport < :1024'           # connections out to well-known ports

And you combine predicates with and / or (also spelled && / ||), which is where this turns into a real query language. Quote the whole expression so the shell leaves your operators alone:

# established connections to a specific host on the HTTPS port
ss -tn 'dst 140.82.113.25 and dport = :443'

# anything talking to port 80 or 443
ss -tn 'dport = :80 or dport = :443'

Combine the two filter languages freely. State filter and address filter sit side by side:

ss -tn state established dst 10.0.0.0/8

That reads as "established TCP connections to the private 10-net", numeric, and it is the sort of thing you would otherwise assemble out of grep pipelines that break the moment an address happens to contain your search string.

Going deeper on a connection

Everything above is about which sockets. The next flags are about knowing one socket in depth, which is where ss quietly becomes a performance tool.

-i shows the kernel's internal TCP information for each connection: the numbers TCP itself is using to decide how fast to send.

$ ss -tin dst 140.82.113.25
State Recv-Q Send-Q Local Address:Port  Peer Address:Port
ESTAB 0      0      192.168.1.50:47320  140.82.113.25:443
	 cubic wscale:7,7 rto:216 rtt:14.5/7.25 mss:1448 cwnd:10 ssthresh:7
	 bytes_sent:8192 bytes_acked:8193 bytes_received:20480 segs_out:24 segs_in:30
	 send 8.0Mbps lastsnd:44 pacing_rate 16.0Mbps delivery_rate 5.2Mbps
	 rcv_rtt:15 rcv_space:14480 retrans:0/2 rcv_ooo:0

That indented block is dense, so here are the fields that earn their keep:

  • cubic the congestion-control algorithm this socket is using. cubic is the modern Linux default; you might also see bbr on tuned hosts.
  • rtt:14.5/7.25 the smoothed round-trip time and its variance, in milliseconds. This is your latency to the peer, measured by the kernel from real traffic rather than by a ping.
  • cwnd:10 the congestion window, in segments. How many segments TCP will keep in flight before waiting for an ACK. This is the single biggest lever on throughput.
  • ssthresh:7 the slow-start threshold. Below it TCP grows the window exponentially; above it, linearly.
  • rto:216 the retransmission timeout in milliseconds: how long TCP waits for an ACK before assuming loss and resending.
  • retrans:0/2 current and total retransmissions. A nonzero total is not alarming on its own, but a climbing total on a connection that should be clean means packet loss on the path.
  • send 8.0Mbps, delivery_rate, pacing_rate the kernel's own estimates of how fast this connection is actually moving data. When someone reports "the download is slow", this line often tells you whether the bottleneck is the window, the loss rate, or genuinely the link.

You do not need to memorize all of it. Knowing that rtt, cwnd, and retrans live here, one command away, changes how you debug a slow connection. It stops being a mystery and becomes three numbers.

-o shows timer information: which kernel timer is armed on the socket and when it fires.

$ ss -ton state established
State Recv-Q Send-Q Local Address:Port Peer Address:Port  Timer
ESTAB 0      0      192.168.1.50:ssh   192.168.1.10:52814 timer:(keepalive,18min,0)

The tuple is (name, time-until-fire, retransmit-count). A keepalive timer is the connection idling; a on (retransmission) timer with a rising count is a connection actively fighting to get data through. Seeing which timer is armed tells you what the socket is currently worried about.

-m shows socket memory usage: how many bytes the kernel has allocated to this socket's buffers.

$ ss -tm state established
ESTAB 0 0 192.168.1.50:ssh 192.168.1.10:52814
	 skmem:(r0,rb131072,t0,tb87040,f0,w0,o0,bl0,d0)

The skmem fields break down receive and send buffer usage. rb and tb are the buffer limits; r and t are current usage; d counts drops. If you are chasing memory pressure from many sockets, this is where the accounting lives.

Summary statistics

-s steps back and prints totals instead of individual sockets. It is the fastest possible answer to "what is the shape of this machine's networking".

$ ss -s
Total: 176
TCP:   8 (estab 4, closed 1, orphaned 0, timewait 1)

Transport Total     IP        IPv6
RAW       1         0         1
UDP       6         4         2
TCP       7         4         3
INET      14        8         6
FRAG      0         0         0

The top line is every socket the kernel holds, across all families. The TCP: line breaks the TCP sockets down by state, which is the health readout I described in the state machine section, available instantly. orphaned is worth knowing: those are sockets with no owning process, usually mid-teardown, and a large orphan count can indicate an application that is closing sockets without draining them.

If you only ever learn two ss commands, make them ss -tlnp for "what is on this box" and ss -s for "how is this box doing".

A few real recipes

The point of learning the pieces is to combine them without thinking. These are the ones I reach for.

What is listening, and who owns it:

ss -tlnp

Count established connections, grouped by the remote host, to find who is hammering you:

ss -tnH state established | awk '{print $4}' | cut -d: -f1 | sort | uniq -c | sort -rn

Count connections by state, the health snapshot as a table:

ss -tanH | awk '{print $1}' | sort | uniq -c | sort -rn

Find the process holding a specific port when "address already in use" is ruining your afternoon:

ss -tlnp 'sport = :8080'

Watch TIME-WAIT accumulation on a busy server, to decide whether you are actually near port exhaustion:

ss -tn state time-wait | wc -l

See only the connections that are stuck somewhere other than established, which surfaces leaks and reachability problems in one glance:

ss -tn exclude established exclude listening

Everyone connected to your SSH port right now, numeric and fast:

ss -tnp 'sport = :22'

Closing

ss rewards the same investment htop does. The bare output looks like noise until you know what each column is telling you, and then the same command becomes a diagnosis. The parts I underused for years turned out to be the valuable ones: Recv-Q and Send-Q on a listening socket, the state aliases like connected and bucket, and the -i block that turns a slow connection from a complaint into three legible numbers.

I did not cover the exotic families (SCTP, MPTCP, vsock, XDP) or the kill switch (-K, which forcibly tears a socket down and which you should be quite sure about before you run). The ss(8) man page is genuinely good and worth a slow read once you have the shape of the tool in your head. Start with ss -tlnp, get curious about a column you have been ignoring, and follow it down.