A lot of times when I'm asked to debug a Unix, Linux, or Mac OS X system, I'll hear a complaint like "The network seems slow", or just "It seems slow", followed by the usual "What is it doing?"
I actually think that last question is a wonderful one: What is this computer doing?
You can see a lot of information about Unix processes using the ps command or the top utility, but it seems like many system administrators don't know how to find networking information, at least not without a network sniffer.
On a Mac OS X system you can find out what servers your Mac is connected to, what ports are in use, and what processes (or applications) are accessing the internet. In this Mac OS X networking tutorial I'll demonstrate how to use the Unix/Linux lsof command on a Mac OS X system to see very cool and helpful network-related information.
Debug Mac OS X network problems with 'lsof'
If you haven't used it before, "lsof" stands for "list open files", and while I normally use it to list the files opened by a particular user or process, it can be used to get great networking information as well.
Diving right in, I now use the Unix/Linux lsof command to find out what ports are in use on a Mac OS X system, as well as which applications are accessing which servers on the internet.
The basic lsof command I use is shown here:
sudo lsof -i -P
Typically I start with this command, and then use other versions of this command if I need other information. Here's some abbreviated, sample networking output from this lsof command:
$ sudo lsof -i -P COMMAND PID USER FD TYPE DEVICE SIZE/OFF NODE NAME launchd 1 root 13u IPv6 0x3cefbe8 0t0 TCP localhost:631 (LISTEN) launchd 1 root 14u IPv4 0x4020e64 0t0 TCP host2.foo.com:631 (LISTEN) ntpd 15 root 20u IPv4 0x3ceba38 0t0 UDP *:123 ntpd 15 root 21u IPv6 0x3ceb6d8 0t0 UDP *:123 ntpd 15 root 22u IPv6 0x3ceccc8 0t0 UDP localhost:123 ntpd 15 root 23u IPv4 0x3cecbf0 0t0 UDP host2.foo.com:123 ntpd 15 root 24u IPv6 0x3cecb18 0t0 UDP localhost:123 ntpd 15 root 25u IPv6 0x3cebb10 0t0 UDP Macintosh-2.local:123 ntpd 15 root 26u IPv4 0x3ceb378 0t0 UDP 192.168.1.101:123 ntpd 15 root 27u IPv6 0x3cec6e0 0t0 UDP Macintosh-2.local:123 mDNSRespo 25 _mdnsresponder 7u IPv4 0x3ceca40 0t0 UDP *:5353 mDNSRespo 25 _mdnsresponder 8u IPv6 0x3cec890 0t0 UDP *:5353 mDNSRespo 25 _mdnsresponder 9u IPv4 0x3ce9368 0t0 UDP *:59492 mDNSRespo 25 _mdnsresponder 11u IPv4 0x3cea0e8 0t0 UDP *:65456 mDNSRespo 25 _mdnsresponder 12u IPv4 0x3ce9878 0t0 UDP *:59834 mDNSRespo 25 _mdnsresponder 13u IPv4 0x3cea7a8 0t0 UDP *:65080 mDNSRespo 25 _mdnsresponder 14u IPv4 0x3ce9950 0t0 UDP *:59946 mDNSRespo 25 _mdnsresponder 15u IPv4 0x3cea880 0t0 UDP *:56912 configd 38 root 9u IPv4 0x3cecda0 0t0 UDP *:* configd 38 root 12u IPv6 0x4641da0 0t0 ICMPV6 *:* configd 38 root 13u IPv6 0x4641e78 0t0 ICMPV6 *:* blued 50 root 3u IPv4 0x3cec380 0t0 UDP *:* prl_dhcpd 72 root 4u IPv4 0x3cec020 0t0 UDP *:* SystemUIS 118 al 9u IPv4 0x3ceb7b0 0t0 UDP *:* iTunes 175 al 21u IPv4 0x61e6270 0t0 TCP *:3689 (LISTEN) iTunes 175 al 22u IPv6 0x3cef984 0t0 TCP *:3689 (LISTEN) ping 4488 al 3u IPv4 0x4641cc8 0t0 ICMP *:* firefox-b 9764 al 30u IPv4 0x3f9366c 0t0 TCP 192.168.1.101:63639->iw-in-f138.1e100.net:80 (ESTABLISHED) firefox-b 9764 al 106u IPv6 0x3cef4bc 0t0 TCP localhost:58884->localhost:58883 (TIME_WAIT) eclipse 11384 al 19u IPv6 0x3cef258 0t0 TCP *:59291 (LISTEN) eclipse 11384 al 20u IPv6 0x3cee664 0t0 TCP localhost:58727->localhost:58726 (TIME_WAIT)
As you can see, this basic command is very nice because it shows the name of the application (the COMMAND column), the process id (PID), user, protocol (TCP, UDP, ICMP), and port and server information. If you've used the Mac OS X netstat command before, you know this is much more information than you can get from netstat.
Mac OS X lsof command options
Here's a quick look at the lsof command options I used in that command:
-i list all internet files -P inhibits conversion of port numbers to port names
As mentioned, I mostly just use those two options, but here is a list of other lsof command arguments you can use related to networking:
-n inhibits conversion of network numbers to host names (makes lsof faster) -N information about NFS files -R show the parent process id (PPID) -T (there are many, many TCP options available; too many to list here) -U Unix domain socket files
In a network debugging application I'm currently writing, I also use the -n option shown here. I've read that this is a good idea when you don't know if there is a network problem, such as a DNS problem, that might cause this command to significantly slow down.
Next, I'll show a few lsof command options that I also use to debug Mac networking issues.
Show only TCP (or UDP, ICMP) with lsof
To show only TCP connections run an lsof command like this:
lsof -i TCP
Similarly, to see all UDP ports, use this command:
lsof -i UDP
Show ports in a LISTEN state with lsof
To see ports in a LISTEN state, use one of the previous commands, and simply grep for LISTEN:
sudo lsof -i -P | grep LISTEN
Or, to look for records in a state of LISTEN, ESTABLISHED, CLOSED, or WAIT, pipe the output into egrep, like this:
sudo lsof -i -P | egrep 'LISTEN|ESTABLISHED|CLOSED|WAIT'
Mac OS X - lsof versus netstat
I think lsof gives you much more information than you can get with the current Mac OS X version of the
netstat command. For instance, here's the output from a netstat command where I try to look at TCP information:
$ netstat -p TCP -a Active Internet connections (including servers) Proto Recv-Q Send-Q Local Address Foreign Address (state) tcp4 0 0 *.59291 *.* LISTEN tcp6 0 0 *.daap *.* LISTEN tcp4 0 0 *.daap *.* LISTEN tcp4 0 0 host2.foo.com.ipp *.* LISTEN tcp6 0 0 localhost.ipp *.* LISTEN
Personally I think this netstat output isn't very useful, and I much prefer the lsof command information that gives you the name of the process, the PID, PPID, and much more.
Other useful Mac OS X lsof commands
Before going, although the following lsof commands aren't directly related to Mac OS X networks, ports, sockets, or the internet, they are very useful whenever you need to know what files are in use by a particular user or application (process).
To see the files opened by a user named alvin, use this command:
lsof -u alvin
To see the files opened by a particular process (such as the "firefox" process shown above), give the lsof command the PID (process ID), like this:
lsof -p 9764
(I promise, you'll be amazed at the files an application like Firefox keeps open at any one time. It currently has a reference to every image file on my Mac Desktop, which strikes me as pretty crazy.)
Internet domain socket states (states you see with lsof or netstat)
As a final note, when debugging OS X network ports, internet connections, etc., it really helps to understand the different states a socket can be in. Here's a list of the possible socket states, courtesy of the netstat man command:
CLOSED: The socket is not in use. LISTEN: The socket is listening for incoming connections. Unconnected listening sockets like these are only displayed when using the -a option. SYN_SENT: The socket is actively trying to establish a connection to a remote peer. SYN_RCVD: The socket has passively received a connection request from a remote peer. ESTABLISHED: The socket has an established connection between a local application and a remote peer. CLOSE_WAIT: The socket connection has been closed by the remote peer, and the system is waiting for the local application to close its half of the connection. LAST_ACK: The socket connection has been closed by the remote peer, the local application has closed its half of the connection, and the system is waiting for the remote peer to acknowledge the close. FIN_WAIT_1: The socket connection has been closed by the local application, the remote peer has not yet acknowledged the close, and the system is waiting for it to close its half of the connection. FIN_WAIT_2: The socket connection has been closed by the local application, the remote peer has acknowledged the close, and the system is waiting for it to close its half of the connection. CLOSING: The socket connection has been closed by the local application and the remote peer simultaneously, and the remote peer has not yet acknowledged the close attempt of the local application. TIME_WAIT: The socket connection has been closed by the local application, the remote peer has closed its half of the connection, and the system is waiting to be sure that the remote peer received the last acknowledgement.
More Mac OS X networking, port, and socket information
I really don't want to oversimplify this networking/port/internet/security topic, so until I can take more time to break it down, I strongly encourage you to read the man pages for the netstat and lsof commands.