My Favorite Unix Tools: A Highly Eclectic Top 10 List
Our resident Unix admin highlights the most treasured utilities in his toolkit.
by James Ervin
8/14/2002 -- As a Unix admin, it's typical to find yourself becoming accustomed to using a small subset of the thousands of utilities any Unix distribution contains. After all, the fun half of Unix administration is cobbling a few tools together to serve an ever-expanding range of esoteric uses. Sometimes, though, I'm in a hurry to complete some slightly off-the-wall task, and find myself thinking that there must be a more appropriate tool, if only I could find it. This is the other half of Unix administration: grinding frustration.
In this article, I'll be presenting some less well-known Unix tools that fill needs you might not have known you had. They're listed with no particular rationale, except that they saved my bacon more than once, and in no particular order, except alphabetical. Some are multi-functional, in the best Unix tradition; others perform just one task, but do it very well. Hopefully at least one or two will be new to your toolbox.
arping
http://www.habets.pp.se/synscan/programs.php?prog=ARPing
Sometimes, you need to know the IP address of a machine that's fouling up the network, but only have the MAC (Media Access Control) number handy. Finding the source of a NetBIOS name conflict on a Windows network is a common cause; especially if the offending machine keeps ducking out of sight. Arping resembles the standard ping utility, but takes a MAC number as input instead of an IP address. There are numerous utilities floating around that perform this function, but this is among the simplest. It won't work on hosts that don't respond to broadcast echo requests, and since disabling responses to such requests can mitigate denial-of-service attacks, arping is useful as a quick litmus test: if you can successfully "arping" a machine, chances are it needs to be secured. Hping, a more advanced utility that can "ping" using a wider range of protocols, is also worth a look.
dsh
http://dsh.sourceforge.net/
The Perl script dsh, short for distributed shell, solves 90 percent of my day-to-day management problems. A distributed shell is simply a method for executing commands on multiple remote machines simultaneously via an interface cosmetically similar to the Unix command line. To accomplish this, an intermediary program that waits for commands must be running on all the target machines. Commercial products such as Network Shell or IBM's own dsh for AIX use a proprietary program as the middleman. The free dsh abstracts this, and allows you to use a remote shell (rsh) or Secure Shell (ssh) server instead. Since dsh is written in Perl, it's also easily extensible and free of charge. If it sounds interesting, take a look at this previous article of mine for some additional information.
dsniff
http://monkey.org/~dugsong/dsniff/
I hesitate even to mention what is potentially the most dangerous utility on this list -- or ever. Dsniff is actually a suite of applications intended to exploit flaws in network protocols: sniffers that capture interesting information from a variety of protocols, utilities that perform man-in-the-middle attacks on SSH and HTTP connections, and other spoofing tools. Improper use of these tools can thoroughly disrupt a network. Properly used, however, dsniff can be a powerful auditing tool, allowing you to sort through the cloud of uninteresting network chatter to see who has weak passwords, who's logging in from suspicious locations, and so forth. Although more advanced networks will be immune to some of dsniff's tricks, the papers and presentations available from the download site are extremely educational, and worth a look even if you never have occasion to use the utilities. "All warfare is based on deception."(Sun Tzu, The Art of War, Project Gutenberg edition)
Fastresolve
http://www.pix.net/staff/djm/sw/fastresolve/
Disabling DNS lookups is one of the first tips in any Web server performance tuning guide. However, this makes post-processing the logs difficult. First, IP addresses are tough to read, so hostnames helps you to group addresses into user populations; second, checking for unregistered IP addresses is worthwhile, since unwillingness to pay for a domain name is a clear hallmark of low character, of course. Fastresolve is a set of utilities that perform massive amounts of DNS lookups quickly and cache the results for later use. I use it to get hostnames for half a gigabyte of Web logs daily in less than an hour. If you manage a Website of any size, or indeed, any service that logs IP addresses instead of hostnames, I recommend checking it or one of the other tools listed here out.
lsof/lslk
ftp://vic.cc.purdue.edu/pub/tools/unix/lsof/
ftp://vic.cc.purdue.edu/pub/tools/unix/lslk/
Lsof and lslk are two older utilities that remain eminently useful. Lsof ("list open files") lists any files currently in use by running processes; lslk ("list locks") lists files that are locked by running processes. Lslk is handy when, for instance, an NFS directory refuses to unmount because a file remains in use -- using lslk, you can discover the file's name, identify the process holding on to it and kill it mercilessly. Lsof, if given the –i switch, can list TCP connections that are open by a process, which is very useful if you can't determine where some of the open connections on your server are originating.
Netcat
http://www.atstake.com/research/tools/
Netcat simply opens arbitrary TCP or UDP connections and outputs arbitrary data. It's a sort of "network swiss army knife," according to its creators. Using the telnet program to connect to an arbitrary TCP port is a quick method of testing connectivity -- for instance, opening a telnet connection to port 25 of a server will let you know if it's delivering mail. However, netcat is more versatile than telnet since it can open UDP connections as well as TCP, thus opening up a wider range of testing possibilities. Also, whereas telnet is limited by its inability to send certain "control" characters, netcat is not. The README file for netcat is worth reading; it's full of imaginative and unusual uses ranging from quick port scanning to bandwidth testing. If netcat sounds interesting, the rest of the tools produced by @stake are also very useful. These are the same folks who produce the Windows password auditing/cracking tool LophtCrack, now known as LC4 -- a strong pedigree.
NetGeo
http://www.caida.org/tools/utilities/netgeo/
NetGeo is one of my favorite utilities, although it is a bit frivolous. It's a Perl module that connects to the freely available NetGeo database maintained by CAIDA, the Cooperative Association for Internet Data Analysis. This database correlates IP addresses and physical locations as closely as is possible. IP address to location mapping is hardly an exact science, since the sources of data, such as the WHOIS database maintained by InterNIC, are infrequently—if ever—updated by domain name registrants. Also, consolidation of Internet providers tends to increase the physical separation of clients and providers -- so a client located in Florida may actually be connecting to the Internet via a provider in St. Louis. Regardless, it's fun to see the results. For instance, the American Midwest is always far more heavily weighted than you might imagine, because many ISPs have backbone installations there. If you have any interest in Internet mapping, I'd suggest mapping the usage of your own services -- if nothing else, you can generate some impressive looking maps. CAIDA also maintains an impressive archive of less frivolous tools for analyzing Internet traffic, including things like graphical traceroute utilities, network mappers, and so on.
Ngrep
http://www.packetfactory.net/Projects/ngrep/
Ngrep is netcat's opposite: Instead of generating arbitrary strings of text on the network, it searches for them. As its name implies, it's simply an application of grep, the ubiquitous regular expression search tool, to the network layer. Give it a string to find, and it will print out every matching packet that passes by. Often, the only way to figure out a network-related application problem is to look at the raw packets, but packets grow too numerous over any but the briefest interval to browse by eye alone. Ngrep eases jobs of this sort considerably, although you have to know what you're looking for in advance, at least partly. Luckily, increasing use of encryption and secure network switches will limit voyeuristic abuses of ngrep (such as searching for the phrase "enter password") in the future.
Ntop
http://www.ntop.org/
The top utility provided an enormously useful function absent from some Unix distributions: a dynamic display of processes, ordered by resource consumption. It's so popular that many people don't even bother to use vendor-provided alternatives when they do exist. Ntop, or network top, is to network usage as top is to processor usage. The text-only version of ntop displays the top bandwidth users by IP address. The graphical version is a full-fledged network protocol analyzer in miniature, complete with embedded web interface. Leave ntop running on monitoring machine on your network, and you'll have cumulative, constantly-updated pie charts dividing your bandwidth by protocol and host. These can be used to pinpoint rogue MP3 servers or prove to your boss that you do, in fact, need more than a 10MB/sec connection to your database server even if it costs a bit more.
scanssh
http://monkey.org/~provos/scanssh/
Like dsniff (above), scanssh comes from the prolific monkey.org. This simple and elegant utility does exactly what it says: Scans a range of network addresses for Secure Shell servers and prints the versions of any it finds. Given the rate at which patches are released for OpenSSH, this tool is useful for keeping any network up-to-date. It's less functional than the more well-known nmap, since it's not a full-fledged network scanner, but is slightly better at identifying SSH versions. 
What are your favorite Unix tools? Share them below!
James Ervin is alone among his coworkers in enjoying Michelangelo Antonioni films, but in his more lucid moments suspects that they're not entirely wrong.
More articles by James Ervin:
|
