This document attempts to list in a step-by-step fashion which network security settings should be changed after installing the server to achieve at least a minimal level of network security. For the sake of simplicity, it is assumed that we want to install a dedicated web server and use SSH for remote administration.
Command line experience is useful, however most of the commands are documented so this document might also serve as a useful example and tutorial on using the shell more effectively.
This document does not:
- Teach you how to edit text. Visual Interactive (vi) is used in the examples but you can use what you like.
- Teach you how to break into other people's systems.
- Teach you how to configure Apache properly.
- Contain instructions for other distributions. Nothing personal here, just lack of time. Most of the information is relevant for other "modern" Linux distributions but some of the commands that are used are RHEL/Centos-specific.
- Tell you how to make your system into a router. Disabling forwarding simplifies the firewalling setup considerably.
- Tell you how to use SELinux effectively. It will try to help you survive with it though.
- Assume that this is all there is to security in general. Security is a large and complex subject out of which network security is just a small part.
- Tell you the complete truth. In order to simplify some things, some facts will be omitted. Do not think that there are no other possibilities in Linux. This document tries to keep things as simple as possible.
That said, if you have suggestions or corrections for this document, feel free to contact me at (czr(at)iki(dot)fi). All contents fall under the regular Copyright, except with the provision that this document can be used for personal (i.e., non-profit) learning/education. For other arrangements, please contact the author (Aleksandr Koltsoff, email given above).
We will start by going through the requirements for our system. We want to setup a Centos4 (or RHEL4) based system in order to serve web pages (either static or with PHP/MySQL and the usual suspects). The system should have good network security but the service must be available publicly (it's a public web server).
The overall process starts with the choices that you can do during installation and right after installation. We then proceed to minimize the security foot print of our services by disabling all the unnecessary services. We then cover the various security mechanisms available for securing the remaining services. We'll also cover other network security mechanisms which do not depend on the specifics of each network service.
Most modern Linux distributions have capable (some might say complex) installation programs which can be used to tailor the system at install phase. In order to save effort after the system has been installed, it makes sense to spend a moment or two while installing. If you have already installed your system, don't worry too much, most of the settings can be also fixed after install, but it will require more effort.
Since going over the whole install process is pretty boring, the phases at which you should stop and think are shown below. Some parts of the installation sequence are omitted on purpose as the issues described in those stages do not directly affect the network security of your system.
[ Selecting a proper software set ]
A custom software set selection is highly recommended since this will allow you to drop everything that is not necessary for a server. You can also select the "Server" profile and customize it later. The screenshot however shows the "Desktop" -profile. Don't use it :-).
[ Protecting the bootloader ]
Most people do not protect the bootloader with a password. And in many cases it does not make sense. You should use a password when your system is located in a physical facility which is not directly controlled by you. One such situation would be where your physical host is located at some insecure server facility.
If you do not use a bootloader password, it will be trivial to gain root access into your system locally. Even if you use a password, it will still be possible to gain root access into your system, but it won't be trivial any more and will require booting your system from other media than the hard disk. If you use a password, also protect your system BIOS so that your system only boots from the hard disk (and not removable media or the network) and set the BIOS password. Root access will then either require the physical removal of hard disks or resetting the CMOS-settings both of which is more easily noticed by staff operating the facility.
For more heavyweight protection, you should consider secure machine facilities and full hard disk encryption, but these are more complex to achieve. Full hard disk encryption is not directly supported by Centos or RHEL but is possible to add manually.
[ Firewall selection ]
Using the kernel firewall (netfilter) will be covered later in this document, but at this point you should enable it and in our case we'll enable ports 22 (ssh), 80 (http) and 443 (https) into our system. Also, leave SELinux at the 'Active' setting. SELinux is a complex security mechanism which has nothing to do with network security (directly) but will be covered briefly at the end of this document.
[ Selecting root password ]
Since this document will cover how to administer your system without knowing the root password, the password selected for the account needs to be "a good one". A good one means a totally random sequence of characters. This sequence is best generated by a specialized program (pwgen is a nice one, but use longer passwords than the default). Your password should contain some special characters as well as be long since size alone doesn't matter. Generate the password, print it on a paper with the hostname and prepare to take the paper to a safe (or other secure physical location). After we setup sudo later on, you will not need the password. It is however a good thing to have for the occasions that you cannot administer the system. For example if you are on a holiday or otherwise unavailable and the system needs to be fixed.
Because of this, the password can be arbitrarily complex and should be long. 15-20 characters is a good length. Try to remember that the system can be accessed even without root password if you have physical access to the system.
[ Useful server configuration tools ]
At some point, you'll have a choice of customizing the software set going into the system. Spend some time here thinking whether you can leave something out. You can install programs later on. Leave out everything that is not important for the service you're implementing. It is recommended to at least install the X window system (you can omit Gnome and KDE if you like) so that the firstboot program will run (covered below). Security vs. graphics will be covered later on.
[ Reducing the number of services ]
After completing the software package customization process, the installation program will continue and after a while your system will boot.
Depending on whether you selected X window system in your software selection, both Centos and RHEL will run a program called firstboot after the first boot. Since the program is graphical, you will not see it if the X server was not installed.
[ Using Network Time Protocol ]
If you are installing on real hardware and not sharing the computer with Windows, select "UTC" for system clock format (not shown in picture). Otherwise leave it at default (non-UTC).
Having a correct system time is very important with relation to security. Since you will want to know the exact time and date when something has happened or something has to happen, a unsynchronized clock in your system will lead into problems. All PC hardware has relatively cheap quartz crystals driving the timer mechanism and this means that after a while, the internal system time will drift away from the real time. Temperature also affects the amount of drift. In short, you need to synchronize your clock from some external source.
Configuring atomic clocks is not covered in this document (they're rather expensive still), but if you google for ntpd documentation, you will find the instructions there. Configuring GPS-based clocks is cheaper, but on the other hand in some cases more difficult since receiving GPS signal indoors requires special arrangements. Getting accurate GPS-time will also require you to acquire or build specialized hardware in order to get the PPS-signal from the receiver. Using an ASCII over RS-232C devices will also work, but their accuracy is lower because of the additional overhead.
Instead of using a real external clock source, we will trust that someone else is using one and try to get the time from them over the network. This is why a protocol called Network Time Protocol was developed and it is also the protocol that we'll use. In order to keep the system time in sync, you'll need to run the ntpd service which will do local time corrections based on external information. The NTP daemon achieves this by asking the kernel to slow down or speed up time. At no point is time "jumped" back or ahead which is important in transactional environments and when using security protocols which rely on synchronized time (Kerberos for example). There is a separate program (ntpdate) that can be used to jump time. This program is normally run automatically at system boot, but after that, ntpd will control time. Note that it is normal for ntpd to fail when it will notice that the local system time is too different from the correct (external) time. It will only start if the current time is close enough for it to attempt corrections. This is why ntpdate is normally run first.
Before installing your system, ask your friendly network administrator for a suitable NTP server to use. Try not to use public servers available over the Internet since some of them are not synchronized at all. Using a restricted set of external NTP servers will also help you to write netfilter firewalling rules. Internet operators normally all have their own servers which you can also use. Using at least two servers will provide redundancy assuming they're both synchronized (which can be tested using the ntpdate program).
Testing time synchronization is covered later on.
A note of warning for people who are running systems within a virtualized hardware environment. Do not attempt to run ntpd in a virtual host. It will not work properly and will actually cause more problems. Instead you need to fix your virtual environment (with a helper program in the host) in order for the system clock to be mirrored from the real clock. In this case your it will be the responsibility of the underlying hardware/operating system to be synchronized. In VMware, you need to install the VMware tools. In Xen, you need not to do anything (except remove ntpd). If you decide not to install the helper program, odds are that your system will either go "too fast" or "too slow" by very large amounts. This is especially true with 2.6 series kernels. In some cases (when using latest and greatest hardware) this will happen even on real hardware. In this case you should contact your hardware manufacturer and either try to find whether they have a newer version of BIOS available or switch to another kernel timer mechanism internally. Google helps in these cases. It is rare for single CPU systems to have these problems, but it is not uncommon to encounter them in multi-core/SMP/NUMA systems.
[ Creating a normal user account ]
During firstboot, it is recommended that you create a normal user account and supply a relatively good password. You will be using this account for the rest of this document. The username user is used in this document but you should select a more descriptive one. Use only lowercase ASCII alphabetical characters in the username.
After you have completed the firstboot program, you will now hopefully see a graphical login screen similar to this one:
[ Graphical login (by gdm) ]
Having a graphical login screen is not recommended for couple of reasons:
- It introduces potential system instability since the X server programs graphic chips directly and sometimes the chip drivers contain bugs. The end result can be a full system
lock-up at the hardware level so you cannot login over the network (or even locally).
- X server is executing with root privileges, which means that it can bypass all security measures in the system. It is a large piece of software and all software contains bugs. In order to minimize the security risk, avoid running it (on a server).
It will run with root privileges even if you login using a normal user account. And since you will never need to login as root, this remains a problem.
For the above reasons it is recommended to disable the graphical login since we can start the graphics environment from the command line when necessary. In this way we minimize the time that X server is running. Some of the administration tools provided by Red Hat are quite nice and useful, and for this reason installing at least the X server environment is recommended. In this document, we'll try our hardest not to use the graphical tools when a suitable text terminal version is available.
Login into the system (via the graphical environment) using your normal user account. Never log in using the root account. There is no need and you want to minimize the number of processes running with root privileges.
[ Clean X environment ]
If you installed Gnome or KDE, your screen will look quite different. Try to locate the terminal emulator and start it in order to get a shell prompt (some people call this "command line").
The next step on our way to a relatively secure system is doing away with the root account. We'll need to do this only once and this is the only time when you need the password for the root account. After this step, you will be able to gain root privileges without knowing the root-password.
[user@centos ~]$ id
uid=500(user) gid=500(user) groups=500(user) context=user_u:system_r:unconfined_t
[user@centos ~]$ su -
Password: Enter the root password for the last time
[root@centos ~]# id
uid=0(root) gid=0(root) groups=0(root),1(bin),2(daemon),3(sys),4(adm),6(disk),10(wheel)
context=root:system_r:unconfined_t
[root@centos ~]# vi /etc/sudoers
[ Starting a shell using root-privileges for the last time ]
At the beginning we're logged in as user and the user belongs only into one group (a group called user, the same as the username). id on RHEL/Centos will also display the SELinux security context in which id is executing.
After this we start a new shell which will run for user root and verify that the shell has the necessary privileges (again by using id).
Our next task is to enable administrative privileges for our normal account. Most Linux distributions come with a program called sudo (Switch User and DO) which is suited for this task. We edit the configuration file for sudo to resemble something like this:
# sudoers file.
#
# This file MUST be edited with the 'visudo' command as root.
#
# See the sudoers man page for the details on how to write a sudoers file.
#
# Host alias specification
# User alias specification
# Cmnd alias specification
# Defaults specification
# User privilege specification
root ALL=(ALL) ALL
# Uncomment to allow people in group wheel to run all commands
# !! NOTE this has been uncommented
%wheel ALL=(ALL) ALL
# Same thing without a password
# %wheel ALL=(ALL) NOPASSWD: ALL
# Samples
# %users ALL=/sbin/mount /cdrom,/sbin/umount /cdrom
# %users localhost=/sbin/shutdown -h now
[ Giving sudo-rights to group called wheel ]
We want to give administrative rights to whomever is in group called wheel. We could also give the rights directly to our user, but using wheel allows more flexibility since we can later on just add users to the wheel group and they will automatically gain superuser rights (via sudo).
Historically the wheel-group is meant for this (long before sudo). On Ubuntu systems the admin group is used for this purpose and sudo has already been setup so that the group is allowed to gain root privileges.
Next we need to add the user to the wheel group and verify that the addition worked.
[root@centos ~]# usermod -a -G wheel user
[root@centos ~]# id user
uid=500(user) gid=500(user) groups=500(user),10(wheel)
[root@centos ~]# exit
[user@centos ~]$ id
uid=500(user) gid=500(user) groups=500(user) context=user_u:system_r:unconfined_t
[ Adding the user to the new group and checking for success ]
You might notice something funny at this stage. According to the id command, user user does indeed now belong to the wheel group. Why then is the result different once we exit the root-shell? Group memberships are set only once when the user logs in. So, we have to logout and login again (easiest). Since we used the graphical login to start with, we need to exit the graphical environment completely (using exit is not enough).
Re-login back using the normal account in order to update the group membership for your session.
We then want to check that the group membership is correct and then test that sudo really works. The easy way to test this is to run a command using sudo that processes a file to which only root has read access. /etc/sudoers is one such file and we count the number of lines in that file using 'word count' (wc). You could also use cat.
[user@centos ~]$ id
uid=500(user) gid=500(user) groups=10(wheel),500(user) context=user_u:system_r:unconfined_t
[user@centos ~]$ sudo wc /etc/sudoers
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these two things:
#1) Respect the privacy of others.
#2) Think before you type.
Password: Type in the password for the normal account.
29 101 614 /etc/sudoers
[ Checking group membership and that sudo works. ]
sudo will be used for the remainder of this document. You might now ask why using sudo is important?
Some reasons for using sudo:
- Each time someone attempts to use sudo, there will be a system log message.
- It is difficult to destroy (by mistake) your system with only normal user access. This is the reason why not to use root account all the time. sudo provides a mechanism so that you can use normal access rights 99% of the time and only temporarily gain root access when you need it.
- You don't have to tell other administrators the root password.
- In a multi-administrator system, the logs become even more important since you can track and blame the correct administrator for breaking something.
- You don't have to re-enter your password each time when using sudo. By default, the system will remember that you have proven to be yourself (via a password) for 5 minutes at a time. This is normally a long enough time to do whatever you want that requires root privileges.
Some distributions force you to use sudo. In those cases, the root account is locked (cannot be used for logging in) and the only way to do something that requires root-privileges is sudo. Ubuntu and Knoppix are good examples of this.
Once you have other administrators on your system you might want to switch to visudo when editing the /etc/sudoers file. visudo will allow only one user to edit the file at a time and will protect against accidental confusion. If you're using a graphical environment and want to execute a graphical program (X client) using root access (not a good idea), you should know that there is a special program called gksudo for this. Too bad it doesn't come with Centos/RHEL by default. In these systems you should start the graphical program within a terminal emulator via sudo.
We are now ready to disable the graphical login. RHEL and Centos implement the graphical login as a separate run level. Run level number 3 is "normal operation" and run level 5 is "same as 3, but with graphical login". So, in order to disable graphical login, we need to change the system default run level into 3. Since run levels are implemented by init, we need to edit its configuration file (/etc/inittab). We only need to change one character on one line, so that the resulting file looks something like this:
#
# inittab This file describes how the INIT process should set up
# the system in a certain run-level.
#
# Author: Miquel van Smoorenburg, <miquels@drinkel.nl.mugnet.org>
# Modified for RHS Linux by Marc Ewing and Donnie Barnes
#
# Default runlevel. The runlevels used by RHS are:
# 0 - halt (Do NOT set initdefault to this)
# 1 - Single user mode
# 2 - Multiuser, without NFS (The same as 3, if you do not have networking)
# 3 - Full multiuser mode
# 4 - unused
# 5 - X11
# 6 - reboot (Do NOT set initdefault to this)
#
# !! NOTE : changed 'id:5' to 'id:3'. This is the only change required.
id:3:initdefault:
# System initialization.
si::sysinit:/etc/rc.d/rc.sysinit
l0:0:wait:/etc/rc.d/rc 0
l1:1:wait:/etc/rc.d/rc 1
l2:2:wait:/etc/rc.d/rc 2
l3:3:wait:/etc/rc.d/rc 3
l4:4:wait:/etc/rc.d/rc 4
l5:5:wait:/etc/rc.d/rc 5
l6:6:wait:/etc/rc.d/rc 6
# Trap CTRL-ALT-DELETE
ca::ctrlaltdel:/sbin/shutdown -t3 -r now
# When our UPS tells us power has failed, assume we have a few minutes
# of power left. Schedule a shutdown for 2 minutes from now.
# This does, of course, assume you have powerd installed and your
# UPS connected and working correctly.
pf::powerfail:/sbin/shutdown -f -h +2 "Power Failure; System Shutting Down"
# If power was restored before the shutdown kicked in, cancel it.
pr:12345:powerokwait:/sbin/shutdown -c "Power Restored; Shutdown Cancelled"
# Run gettys in standard runlevels
1:2345:respawn:/sbin/mingetty tty1
2:2345:respawn:/sbin/mingetty tty2
3:2345:respawn:/sbin/mingetty tty3
4:2345:respawn:/sbin/mingetty tty4
5:2345:respawn:/sbin/mingetty tty5
6:2345:respawn:/sbin/mingetty tty6
# Run xdm in runlevel 5
x:5:respawn:/etc/X11/prefdm -nodaemon
[ New init default is now 3. ]
[user@centos ~]$ sudo vi /etc/inittab
Password: Enter your own password
[user@centos ~]$ runlevel
bash: runlevel: command not found
[user@centos ~]$ /sbin/runlevel
N 5
[user@centos ~]$ /sbin/init 1
init: must be superuser.
[user@centos ~]$ sudo /sbin/init 1
[ Editing the file and testing init ]
After editing the file, we next try out couple of init-related commands. We first try to show the current system runlevel by using the runlevel command. It will display two characters, the one on the right is the current runlevel.
You will notice that the command is not in the command path of a normal user. This doesn't mean that we cannot run it. If we use the absolute path, we can attempt at running it. The command will complain when we don't have the necessary rights (as can be seen). Both of these cases will be seen in the following examples.
We change the system to run level 1 (single-user mode) without any good reason. If you're doing this over the network (using SSH), you will be kicked out of the system since there is no networking in the single-user mode. Mind where you step.
Telling INIT to go to single user mode.
INIT: Going single user
INIT: Sending processes the TERM signal
sh-3.00# init 6
[ In single-user mode and rebooting ]
The only proper way to test the default run level is to reboot the system. You can use the reboot command (switches the system to run level 6), press Ctrl+Alt+Delete (if in text mode). This will also switch the system to run level 6. Finally, some people like to use shutdown -r, which also will switch into run level 6. Run level 6 will shutdown all services in a controlled fashion and finally start a program that will ask the kernel to do a hardware reset.
So, after rebooting, you should now have a nice prompt from login running on a Linux virtual terminal 1:
[ Clean text mode environment ]
At this stage (at your option), you may start your graphical environment using the startx script. If you have installed Gnome/KDE, the default graphical environment will be started. Try to remember not to run startx as root. It's still not a good idea.
The next step is to find and stop unnecessary services from starting. Note that removing the software packages for the services is also an option, but at this stage we assume that you didn't install anything unnecessary. As you will see, there is still plenty to do.
We'll start by getting a list of services or service-like things in our system. Red Hat provides a nice program for this called chkconfig. It is used to list and enable/disable services at various runlevels. Since we only care whether a service will start or not (and not the particular run levels), it will be quite easy to use.
[user@centos ~]$ chkconfig --list
bash: chkconfig: command not found
[user@centos ~]$ /sbin/chkconfig --list
psacct 0:off 1:off 2:off 3:off 4:off 5:off 6:off
nfs 0:off 1:off 2:off 3:off 4:off 5:off 6:off
nscd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
readahead_early 0:off 1:off 2:off 3:off 4:off 5:on 6:off
isdn 0:off 1:off 2:on 3:on 4:on 5:on 6:off
portmap 0:off 1:off 2:off 3:on 4:on 5:on 6:off
autofs 0:off 1:off 2:off 3:on 4:on 5:on 6:off
winbind 0:off 1:off 2:off 3:off 4:off 5:off 6:off
rawdevices 0:off 1:off 2:off 3:on 4:on 5:on 6:off
irqbalance 0:off 1:off 2:off 3:on 4:on 5:on 6:off
dc_server 0:off 1:off 2:off 3:off 4:off 5:off 6:off
httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
xinetd 0:off 1:off 2:off 3:on 4:on 5:on 6:off
pcmcia 0:off 1:off 2:on 3:on 4:on 5:on 6:off
iptables 0:off 1:off 2:on 3:on 4:on 5:on 6:off
acpid 0:off 1:off 2:off 3:on 4:on 5:on 6:off
bluetooth 0:off 1:off 2:off 3:off 4:off 5:off 6:off
mdmpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
cpuspeed 0:off 1:on 2:on 3:on 4:on 5:on 6:off
ipmi 0:off 1:off 2:off 3:off 4:off 5:off 6:off
auditd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
sendmail 0:off 1:off 2:on 3:on 4:on 5:on 6:off
cups-config-daemon 0:off 1:off 2:off 3:on 4:on 5:on 6:off
readahead 0:off 1:off 2:off 3:off 4:off 5:on 6:off
messagebus 0:off 1:off 2:off 3:on 4:on 5:on 6:off
arptables_jf 0:off 1:off 2:on 3:on 4:on 5:on 6:off
saslauthd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
diskdump 0:off 1:off 2:off 3:off 4:off 5:off 6:off
yum 0:off 1:off 2:off 3:off 4:off 5:off 6:off
mdmonitor 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rpcgssd 0:off 1:off 2:off 3:on 4:on 5:on 6:off
haldaemon 0:off 1:off 2:off 3:on 4:on 5:on 6:off
cups 0:off 1:off 2:on 3:on 4:on 5:on 6:off
microcode_ctl 0:off 1:off 2:on 3:on 4:on 5:on 6:off
kudzu 0:off 1:off 2:off 3:on 4:on 5:on 6:off
sshd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
tux 0:off 1:off 2:off 3:off 4:off 5:off 6:off
dc_client 0:off 1:off 2:off 3:off 4:off 5:off 6:off
ibmasm 0:off 1:off 2:off 3:off 4:off 5:off 6:off
syslog 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rpcidmapd 0:off 1:off 2:off 3:on 4:on 5:on 6:off
network 0:off 1:off 2:on 3:on 4:on 5:on 6:off
NetworkManager 0:off 1:off 2:off 3:off 4:off 5:off 6:off
openibd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
anacron 0:off 1:off 2:on 3:on 4:on 5:on 6:off
xfs 0:off 1:off 2:on 3:on 4:on 5:on 6:off
atd 0:off 1:off 2:off 3:on 4:on 5:on 6:off
crond 0:off 1:off 2:on 3:on 4:on 5:on 6:off
smartd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rhnsd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
ypbind 0:off 1:off 2:off 3:off 4:off 5:off 6:off
netfs 0:off 1:off 2:off 3:on 4:on 5:on 6:off
ntpd 0:off 1:off 2:off 3:on 4:off 5:on 6:off
netdump 0:off 1:off 2:off 3:off 4:off 5:off 6:off
irda 0:off 1:off 2:off 3:off 4:off 5:off 6:off
apmd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
nfslock 0:off 1:off 2:off 3:on 4:on 5:on 6:off
gpm 0:off 1:off 2:on 3:on 4:on 5:on 6:off
netplugd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
xinetd based services:
eklogin: off
rsync: off
chargen: off
krb5-telnet: off
gssftp: off
klogin: off
daytime-udp: off
time-udp: off
echo-udp: off
echo: off
time: off
cups-lpd: off
kshell: off
chargen-udp: off
daytime: off
[user@centos ~]$ /sbin/service gpm status
gpm (pid 2583) is running...
[user@centos ~]$ /sbin/service gpm stop
rm: cannot remove `/var/run/gpm.pid': Permission denied [FAILED]
rm: cannot remove `/var/lock/subsys/gpm': Permission denied
[user@centos ~]$ sudo /sbin/service gpm stop
Shutting down console mouse services: [ OK ]
[user@centos ~]$ /sbin/chkconfig gpm off
failed to make symlink /etc/rc2.d/K15gpm: Permission denied
failed to make symlink /etc/rc3.d/K15gpm: Permission denied
failed to make symlink /etc/rc4.d/K15gpm: Permission denied
failed to make symlink /etc/rc5.d/K15gpm: Permission denied
[user@centos ~]$ sudo /sbin/chkconfig gpm off
[ Listing and controlling services ]
Don't be intimidated by the long list of services. Most of them are not actually network services, but instead some scripts that are executed when entering a run level or starting the system. Most of them do not leave daemon processes running on the background. As you can also see, a lot of them are already off on all runlevels. This is good since we don't have to do so much.
As an example of controlling a service without affecting its on/off status we check the status of gpm. gpm is a small daemon that allows you to use the mouse in Linux virtual terminals in text-mode. Copy/paste works the same way as in X and also between different virtual terminals. Since we're not going to use the mouse when locally at the computer (it's a server after all), we'll start by disabling the service. Disabling the service startup does not affect the current running status of the service, and this is why we need both service and chkconfig. Later you'll see a more generic way of using service scripts (instead of service). Both service and chkconfig are RH/Centos-specific commands and on other distributions you'll have to use something else instead.
It would also be nice for the service scripts to check whether they're run with root-privileges, but as you can see, they don't. This leads to a lot of error messages, which are harmless.
We now need to disable those services that we don't need. Sounds easy? It is once you know what each service is meant for. In order to get this information we'll have to consult the software package database and read the description of the packages that introduce the service scripts into our system. Let's take couple of service names that interest us and assume that they come in a package whose name is identical to the service name. We'll also see how to determine the package name if the service name doesn't correspond to a package name.
[user@centos ~]$ rpm -q psacct
psacct-6.3.2-38.rhel4
[user@centos ~]$ rpm -qi psacct
Name : psacct Relocations: (not relocatable)
Version : 6.3.2 Vendor: CentOS
Release : 38.rhel4 Build Date: Wed 08 Mar 2006 04:00:47 PM EET
Install Date: Mon 16 Oct 2006 07:13:35 PM EEST Build Host: build-i386
Group : Applications/System Source RPM: psacct-6.3.2-38.rhel4.src.rpm
Size : 89524 License: GPL
Signature : DSA/SHA1, Thu 09 Mar 2006 06:06:51 AM EET, Key ID a53d0bab443e1821
Packager : Johnny Hughes <johnny@centos.org>
Summary : Utilities for monitoring process activities.
Description :
The psacct package contains several utilities for monitoring process
activities, including ac, lastcomm, accton and sa. The ac command
displays statistics about how long users have been logged on. The
lastcomm command displays information about previous executed
commands. The accton command turns process accounting on or off. The
sa command summarizes information about previously executed
commmands.
[user@centos ~]$ rpm -q microcode_ctl
package microcode_ctl is not installed
[user@centos ~]$ ls -l /etc/init.d/microcode_ctl
-rwxr-xr-x 1 root root 1731 Aug 13 10:15 /etc/init.d/microcode_ctl
[user@centos ~]$ rpm -qf /etc/init.d/microcode_ctl
kernel-utils-2.4-13.1.83
[user@centos ~]$ rpm -qi kernel-utils
Name : kernel-utils Relocations: (not relocatable)
Version : 2.4 Vendor: CentOS
Release : 13.1.83 Build Date: Sun 13 Aug 2006 10:15:31 AM EEST
Install Date: Mon 16 Oct 2006 07:13:48 PM EEST Build Host: build-i386
Group : System Environment/Base Source RPM: kernel-utils-2.4-13.1.83.src.rpm
Size : 1558679 License: GPL/OSL
Signature : DSA/SHA1, Sun 13 Aug 2006 03:55:20 PM EEST, Key ID a53d0bab443e1821
Packager : Johnny Hughes <johnny@centos.org>
Summary : Kernel and Hardware related utilities
Description :
kernel-utils contains several utilities that can be used to control
the kernel or your machines hardware. Included are
* cpuspeed - dynamically change the speed of CPUs (if CPU is capable)
* dmidecode - gives information about the bios and motherboard revisions
* irqbalance - Evenly distribute interrupt load across CPUs.
* microcode_ctl - updates the microcode on Intel cpus
* rng-tools - Hardware random number generation tools.
* smartctl - monitor the health of your disks
[user@centos ~]$ man microcode_ctl
[user@centos ~]$ rpm -qd kernel-utils
/usr/share/doc/smartmontools-5.33/AUTHORS
/usr/share/doc/smartmontools-5.33/CHANGELOG
/usr/share/doc/smartmontools-5.33/COPYING
/usr/share/doc/smartmontools-5.33/INSTALL
/usr/share/doc/smartmontools-5.33/NEWS
/usr/share/doc/smartmontools-5.33/README
/usr/share/doc/smartmontools-5.33/TODO
/usr/share/doc/smartmontools-5.33/WARNINGS
/usr/share/doc/smartmontools-5.33/examplescripts/Example1
/usr/share/doc/smartmontools-5.33/examplescripts/Example2
/usr/share/doc/smartmontools-5.33/examplescripts/Example3
/usr/share/doc/smartmontools-5.33/examplescripts/README
/usr/share/doc/smartmontools-5.33/smartd.conf
/usr/share/man/man1/hardlink.1.gz
/usr/share/man/man1/irqbalance.1.gz
/usr/share/man/man1/longrun.1.gz
/usr/share/man/man1/rngtest.1.gz
/usr/share/man/man1/x86info.1.gz
/usr/share/man/man5/smartd.conf.5.gz
/usr/share/man/man8/microcode_ctl.8.gz
/usr/share/man/man8/rngd.8.gz
/usr/share/man/man8/smartctl.8.gz
/usr/share/man/man8/smartd.8.gz
[ Finding out what the different services do ]
Here's a short explanation for the commands:
- rpm -q packagename : query the database whether there is a package installed with the given name.
- rpm -qi packagename : display short information about the package.
- rpm -qf /path/to/a/file : display the package name to who the given file belongs (most files belong to some package).
- man name-of-command : if you don't know this, then ask someone to implement network security for you.
- rpm -qd packagename : list any files from package that have been marked as "documentation". So that you may read them for more information. The filenames ending with .[1-9].gz are manual pages.
After you know which services you don't need, you need to disable them. In the following example this is done with a small for-loop in the shell, but unless you're comfortable with shell, you should probably disable each service with a separate command.
Note that it would be sensible also to execute each of the service scripts with the stop parameter. We won't do that here since we will be rebooting the system at some point so that the services won't start. If you do not plan to reboot your system after this step, also stop the services, don't just disable them from the run levels!
After disabling the services, we verify that the ones that are left enabled are the necessary ones.
[user@centos ~]$ for a in apmd atd openibd rpcidmapd kudzu \
microcode_ctl cups rpcgssd arptables_jf cups-config-daemon \
pcmcia isdn; do sudo /sbin/chkconfig $a off; done
Password: Some time elapsed since last authentication, enter your password
[user@centos ~]$ /sbin/chkconfig --list
psacct 0:off 1:off 2:off 3:off 4:off 5:off 6:off
nfs 0:off 1:off 2:off 3:off 4:off 5:off 6:off
nscd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
readahead_early 0:off 1:off 2:off 3:off 4:off 5:on 6:off
isdn 0:off 1:off 2:off 3:off 4:off 5:off 6:off
portmap 0:off 1:off 2:off 3:on 4:on 5:on 6:off
autofs 0:off 1:off 2:off 3:on 4:on 5:on 6:off
winbind 0:off 1:off 2:off 3:off 4:off 5:off 6:off
rawdevices 0:off 1:off 2:off 3:on 4:on 5:on 6:off
irqbalance 0:off 1:off 2:off 3:on 4:on 5:on 6:off
dc_server 0:off 1:off 2:off 3:off 4:off 5:off 6:off
httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
xinetd 0:off 1:off 2:off 3:on 4:on 5:on 6:off
pcmcia 0:off 1:off 2:off 3:off 4:off 5:off 6:off
iptables 0:off 1:off 2:on 3:on 4:on 5:on 6:off
acpid 0:off 1:off 2:off 3:on 4:on 5:on 6:off
bluetooth 0:off 1:off 2:off 3:off 4:off 5:off 6:off
mdmpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
cpuspeed 0:off 1:on 2:on 3:on 4:on 5:on 6:off
ipmi 0:off 1:off 2:off 3:off 4:off 5:off 6:off
auditd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
sendmail 0:off 1:off 2:on 3:on 4:on 5:on 6:off
cups-config-daemon 0:off 1:off 2:off 3:off 4:off 5:off 6:off
readahead 0:off 1:off 2:off 3:off 4:off 5:on 6:off
messagebus 0:off 1:off 2:off 3:on 4:on 5:on 6:off
arptables_jf 0:off 1:off 2:off 3:off 4:off 5:off 6:off
saslauthd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
diskdump 0:off 1:off 2:off 3:off 4:off 5:off 6:off
yum 0:off 1:off 2:off 3:off 4:off 5:off 6:off
mdmonitor 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rpcgssd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
haldaemon 0:off 1:off 2:off 3:on 4:on 5:on 6:off
cups 0:off 1:off 2:off 3:off 4:off 5:off 6:off
microcode_ctl 0:off 1:off 2:off 3:off 4:off 5:off 6:off
kudzu 0:off 1:off 2:off 3:off 4:off 5:off 6:off
sshd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
tux 0:off 1:off 2:off 3:off 4:off 5:off 6:off
dc_client 0:off 1:off 2:off 3:off 4:off 5:off 6:off
ibmasm 0:off 1:off 2:off 3:off 4:off 5:off 6:off
syslog 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rpcidmapd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
network 0:off 1:off 2:on 3:on 4:on 5:on 6:off
NetworkManager 0:off 1:off 2:off 3:off 4:off 5:off 6:off
openibd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
anacron 0:off 1:off 2:on 3:on 4:on 5:on 6:off
xfs 0:off 1:off 2:on 3:on 4:on 5:on 6:off
atd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
crond 0:off 1:off 2:on 3:on 4:on 5:on 6:off
smartd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
rhnsd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
ypbind 0:off 1:off 2:off 3:off 4:off 5:off 6:off
netfs 0:off 1:off 2:off 3:on 4:on 5:on 6:off
ntpd 0:off 1:off 2:off 3:on 4:off 5:on 6:off
netdump 0:off 1:off 2:off 3:off 4:off 5:off 6:off
irda 0:off 1:off 2:off 3:off 4:off 5:off 6:off
apmd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
nfslock 0:off 1:off 2:off 3:on 4:on 5:on 6:off
gpm 0:off 1:off 2:off 3:off 4:off 5:off 6:off
netplugd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
xinetd based services:
eklogin: off
rsync: off
chargen: off
krb5-telnet: off
gssftp: off
klogin: off
daytime-udp: off
time-udp: off
echo-udp: off
echo: off
time: off
cups-lpd: off
kshell: off
chargen-udp: off
daytime: off
[user@centos ~]$ sudo /sbin/chkconfig xinetd off
[user@centos ~]$ sudo /sbin/chkconfig --list xinetd
xinetd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[user@centos ~]$ sudo reboot
[ Disabling unnecessary services en-masse ]
We also disable the xinetd service since we do not need any of the subservices it seems to provide (xinetd is covered later on). Finally, in order to test that the system still boots up properly, we reboot.
If your system doesn't boot properly, you'll have to boot it in single-user mode and re-enable the services that are critical after all. If you manage to trash your system by disabling critical services, you probably didn't read what each of the services does. The example above only demonstrates how to query information for two services. You should query for all the services that you plan to disable. Using the single-user mode is not covered in this document.
Before continuing too far along the road, this is a good place to check whether NTP is working. This is done to verify that our system has the correct time and the logs will contain correct timestamps.
In order to do this we'll use the ntpdate program. First in debug mode which doesn't change the system clock, but is useful to check what the clock offset would be, and then running it again to set the system clock. In this case, the machine is running under VMware, and the support tools have not been installed. System clock is running about twice as fast compared to real time. Using ntpd will not help.
When using ntpdate with the -d option, it goes through the whole querying process in verbose mode but does not affect the system time. Note that last line of ntpdate output ("step time server"). This gives you the amount of seconds that your system clock differs from the external NTP source.
When running without -d, ntpdate will be considerably more quiet and modifies the system clock if it gets any time from the external source. You will see the same step time server line as before. The reason why there's such a large difference between the two offsets in this case is because of VMware and the missing vm-utilities.
[user@centos ~]$ sudo /usr/sbin/ntpdate -d time.intranet
17 Oct 14:55:31 ntpdate[3299]: ntpdate 4.2.0a@1.1190-r Sun Aug 13 01:49:13 CDT 2006 (1)
Looking for host time.intranet and service ntp
transmit(192.168.1.4)
receive(192.168.1.4)
transmit(192.168.1.4)
receive(192.168.1.4)
transmit(192.168.1.4)
receive(192.168.1.4)
transmit(192.168.1.4)
receive(192.168.1.4)
transmit(192.168.1.4)
server 192.168.1.4, port 123
stratum 3, precision -18, leap 00, trust 000
refid [192.168.1.4], delay 0.02663, dispersion 0.00143
transmitted 4, in filter 4
reference time: c8df4292.74bc8c0c Tue, Oct 17 2006 14:52:50.456
originate timestamp: c8df42a5.5bd102bc Tue, Oct 17 2006 14:53:09.358
transmit timestamp: c8df4333.73ca1059 Tue, Oct 17 2006 14:55:31.452
filter delay: 0.02757 0.02730 0.02663 0.02841
0.00000 0.00000 0.00000 0.00000
filter offset: -142.090 -142.092 -142.093 -142.095
0.000000 0.000000 0.000000 0.000000
delay 0.02663, dispersion 0.00143
offset -142.093532
17 Oct 14:55:31 ntpdate[3299]: step time server 192.168.1.4 offset -142.093532 sec
[user@centos ~]$ sudo /usr/sbin/ntpdate time.intranet
17 Oct 14:56:34 ntpdate[3328]: the NTP socket is in use, exiting
[user@centos ~]$ sudo /etc/init.d/ntpd stop
Shutting down ntpd: [ OK ]
[user@centos ~]$ sudo /usr/sbin/ntpdate time.intranet
17 Oct 14:54:13 ntpdate[3338]: step time server 192.168.1.4 offset -160.541355 sec
[user@centos ~]$ sudo /etc/init.d/ntpd start
ntpd: Synchronizing with time server: [ OK ]
Starting ntpd: [ OK ]
[ Checking whether NTP is working ]
We are now ready to proceed with network security. First thing that we should start with is getting a reliable map about the network visibility of our host. In order to do this, the netstat network statistics tool is most useful.
We'll make multiple listings of active TCP-connections (-t) and listings of sockets which can be connected to. Note that at this point we don't yet care whether they can be connected from outside of our system or not.
In fact, what we're really still doing is finding out whether there are still some services running which we don't need and might cause extra risk. The technique applied here is relevant to similar cases when you want to know what is the function of some process which is using network (either as a service or a client).
We'll first present the listing in which we'll locate and eliminate couple of network services. We'll also start our web server (as an example). Please note that the netstat listings have been edited for better formatting.
[user@centos ~]$ netstat -t
Active Internet connections (w/o servers)
Pro Local Address Foreign Address State
tcp centos.intranet:ssh itchy.intranet:56231 ESTABLISHED
[user@centos ~]$ netstat -tn
Active Internet connections (w/o servers)
Pro Local Address Foreign Address State
tcp ::ffff:192.168.1.30:22 ::ffff:192.168.1.3:56231 ESTABLISHED
[user@centos ~]$ netstat -tl
Active Internet connections (only servers)
Pro Local Address Foreign Address State
tcp *:sunrpc *:* LISTEN
tcp localhost.localdomain:smtp *:* LISTEN
tcp *:1018 *:* LISTEN
tcp *:ssh *:* LISTEN
[user@centos ~]$ netstat -tln
Active Internet connections (only servers)
Pro Local Address Foreign Address State
tcp 0.0.0.0:111 0.0.0.0:* LISTEN
tcp 127.0.0.1:25 0.0.0.0:* LISTEN
tcp 0.0.0.0:1018 0.0.0.0:* LISTEN
tcp :::22 :::* LISTEN
[user@centos ~]$ sudo netstat -tlnp
Active Internet connections (only servers)
Pro Local Address Foreign Address State PID/Program name
tcp 0.0.0.0:111 0.0.0.0:* LISTEN 2089/portmap
tcp 127.0.0.1:25 0.0.0.0:* LISTEN 2339/sendmail: acce
tcp 0.0.0.0:1018 0.0.0.0:* LISTEN 2108/rpc.statd
tcp :::22 :::* LISTEN 2216/sshd
[user@centos ~]$ sudo netstat -ulnp
Active Internet connections (only servers)
Pro Local Address Foreign Address PID/Program name
udp 0.0.0.0:68 0.0.0.0:* 2019/dhclient
udp 0.0.0.0:111 0.0.0.0:* 2089/portmap
udp 0.0.0.0:1012 0.0.0.0:* 2108/rpc.statd
udp 0.0.0.0:1015 0.0.0.0:* 2108/rpc.statd
udp 192.168.1.30:123 0.0.0.0:* 2320/ntpd
udp 127.0.0.1:123 0.0.0.0:* 2320/ntpd
udp 0.0.0.0:123 0.0.0.0:* 2320/ntpd
udp :::123 :::* 2320/ntpd
[user@centos ~]$ rpm -q rpc.statd
package rpc.statd is not installed
[user@centos ~]$ cd /proc/2108
[user@centos 2108]$ ls -la exe
ls: cannot read symbolic link exe: Permission denied
lrwxrwxrwx 1 root root 0 Oct 16 20:49 exe
[user@centos 2108]$ sudo ls -la exe
lrwxrwxrwx 1 root root 0 Oct 16 20:49 exe ->> /sbin/rpc.statd
[user@centos 2108]$ rpm -qf /sbin/rpc.statd
nfs-utils-1.0.6-70.EL4
[user@centos 2108]$ rpm -qi nfs-utils
Name : nfs-utils Relocations: (not relocatable)
Version : 1.0.6 Vendor: CentOS
Release : 70.EL4 Build Date: Sun 13 Aug 2006 09:52:53 AM EEST
Install Date: Mon 16 Oct 2006 07:13:56 PM EEST Build Host: build-i386
Group : System Environment/Daemons Source RPM: nfs-utils-1.0.6-70.EL4.src.rpm
Size : 697509 License: GPL
Signature : DSA/SHA1, Sun 13 Aug 2006 03:55:26 PM EEST, Key ID a53d0bab443e1821
Packager : Johnny Hughes <johnny@centos.org>>
Summary : NFS utlilities and supporting daemons for the kernel NFS server.
Description :
The nfs-utils package provides a daemon for the kernel NFS server and
related tools, which provides a much higher level of performance than the
traditional Linux NFS server used by most users.
This package also contains the showmount program. Showmount queries the
mount daemon on a remote host for information about the NFS (Network File
System) server on the remote host. For example, showmount can display the
clients which are mounted on that host.
[user@centos 2108]$ rpm -qi portmap
Name : portmap Relocations: (not relocatable)
Version : 4.0 Vendor: CentOS
Release : 63 Build Date: Mon 21 Feb 2005 06:54:02 PM EET
Install Date: Mon 16 Oct 2006 07:13:55 PM EEST Build Host: bhrama.build.karan.org
Group : System Environment/Daemons Source RPM: portmap-4.0-63.src.rpm
Size : 53261 License: BSD
Signature : DSA/SHA1, Sat 26 Feb 2005 11:34:55 PM EET, Key ID a53d0bab443e1821
Packager : Karanbir Singh <kbsingh@centos.org>>
Summary : A program which manages RPC connections.
Description :
The portmapper program is a security tool which prevents theft of NIS
(YP), NFS and other sensitive information via the portmapper. A
portmapper manages RPC connections, which are used by protocols like
NFS and NIS.
The portmap package should be installed on any machine which acts as a
server for protocols using RPC.
[user@centos 2108]$ cd
[user@centos ~]$ /sbin/chkconfig --list | grep portmap
portmap 0:off 1:off 2:off 3:on 4:on 5:on 6:off
[user@centos ~]$ /sbin/chkconfig --list | grep nfs
nfs 0:off 1:off 2:off 3:off 4:off 5:off 6:off
nfslock 0:off 1:off 2:off 3:on 4:on 5:on 6:off
[user@centos ~]$ /sbin/chkconfig --list | grep netfs
netfs 0:off 1:off 2:off 3:on 4:on 5:on 6:off
[user@centos ~]$ /sbin/chkconfig --list | grep autofs
autofs 0:off 1:off 2:off 3:on 4:on 5:on 6:off
[user@centos ~]$ sudo /sbin/service portmap stop
Stopping portmap: [ OK ]
[user@centos ~]$ sudo /sbin/service nfslock stop
Stopping NFS statd: [ OK ]
[user@centos ~]$ sudo /sbin/service netfs stop
[user@centos ~]$ sudo /sbin/service autofs stop
Stopping automount: [ OK ]
[user@centos ~]$ netstat -uln
Active Internet connections (only servers)
Pro Local Address Foreign Address
udp 0.0.0.0:68 0.0.0.0:*
udp 192.168.1.30:123 0.0.0.0:*
udp 127.0.0.1:123 0.0.0.0:*
udp 0.0.0.0:123 0.0.0.0:*
udp :::123 :::*
[user@centos ~]$ for a in portmap nfslock netfs autofs; do
sudo /sbin/chkconfig $a off; done
[user@centos ~]$ /sbin/chkconfig --list httpd
httpd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[user@centos ~]$ sudo /sbin/service httpd start
Starting httpd: [ OK ]
[user@centos ~]$ sudo netstat -tlp
Active Internet connections (only servers)
Pro Local Address Foreign Address PID/Program name
tcp localhost.localdomain:smtp *:* 2339/sendmail: acce
tcp *:http *:* 3579/httpd
tcp *:ssh *:* 2216/sshd
tcp *:https *:* 3579/httpd
[ Doing local analysis of network attack vectors ]
Short explanations for new commands:
- netstat -t : show active TCP sockets on our system. In this case we see that there is one connection to the SSH service running on our system.
- netstat -tn : same as -t but use numbers in report instead of symbolic meanings. Both forms are useful in practice.
- netstat -tl : show TCP sockets which can be connected to (TCP network services).
- netstat -tlnp : also display process ID and process name that is keeping the socket open.
- netstat -u : display UDP sockets. They cannot have connections, but it's possible to have UDP servers (as you can see).
- cd /proc/PID/ : switch to the process information directory of process PID.
- /proc/PID/exe : symbolic link to the file from which the process has started. Most people will understand this as "the program running".
We dissect the listing above. We start by checking if we have any existing TCP connections to and from any address (including our own). We then move onto listing the sockets that are in listening state and notice that there are couple of TCP ports which we're not familiar with. That is why we want to see which processes are keeping the ports open (portmap and rpc.statd). We also notice that there are some processes on the UDP-side of things.
We try to get more information about the program, but since it's not directly a package name (rpm doesn't find it) we proceed with a much more powerful technique. Namely, we need to get a filename which to use with rpm -qf. You can get the filename of the program that is running by consulting the exe symbolic link under each processes proc-directory. Since we don't own the process in question, we'll need root access to get the contents of the symlink.
After that, we find out what each of the programs are designed to do (by reading the package descriptions) and also that the person writing the short descriptions at Red Hat needs a spelling checker (as does the author of this document).
We also list some information about related packages (from past experience) and decide that we don't need to access servers using NFS, nor provide NFS services. We're setting up a web server, so we disable all the NFS-related services. We also verify that Apache (web server) starts succesfully by checking that the port is listened using netstat.
At this point we have reduced our network visibility. We still have to add security mechanisms to the remaining attack vectors and this is where the real fun begins.
Linux like many other modern UNIX-systems provides several network security mechanisms which are layered on top of each other.
Depending on the service in question you may:
- Control network related security from the service configuration file. This is quite common for software which is complex and large.
Examples: Apache web server (httpd), Squid Internet proxy (squid), FTP server (vsftpd), CUPS (network printing system) and so on.
- A service might also utilize a library called "TCP Wrappers" (covered later) in which case you can also use the wrappers to provide additional restrictions and rules.
- For very simple services (in terms of software complexity), it might support (or require) execution under an internet super daemon (xinetd on Centos/RHEL systems). In this case the software itself probably doesn't support access control lists or other security mechanisms, but xinetd already has them. On Centos/RHEL systems xinetd uses TCP Wrappers as well, so you have two layers at which to implement security.
Irrespective of the above mechanisms, you can always use the kernel network firewall called netfilter. Each level of security has a different set of features and feature restrictions, so it might be necessary to set security settings at multiple levels for one service. This is what we'll eventually do as well.
Below is a simplified model of the network security layers available:
[ Overview of network security layers ]
Click here for full resolution version (new window)
Processes that require root privileges are shown in orange. Normal processes in green (they might require root for other reasons though).
One point of note in the above picture is the process called "attack tool". If you have root access on a Linux system, you can bypass all the security layers by using raw packet interfaces. Needless to say, this is quite dangerous as this will allow one to generate forged traffic and capture all traffic coming to the host. The capturing interface is normally used by network sniffer programs (tcpdump and ethereal, which has been renamed to wireshark).
This document does not cover attack tools, but you should bear in mind that even if you implement network security at all the above levels, you'll still need to implement other security mechanisms.
Having the possibility to connect over the network to do remote administration is certainly a nice feature. With this feature comes a security risk however. The risk is greater if you need to allow SSH from anywhere across Internet.
The main security risk comes from the fact that the SSH server (OpenSSH) requires root access to execute. There are two reasons for this, one is binding to port number less than 1024 (old UNIX "security" feature). You can change the port on which SSH will listen from the configuration file, so this could be fixed. However, the other reason cannot be easily fixed. In order to start a shell as a logged in user, the process needs to switch UID first. In order to switch the UID, the process needs to execute as root. There is no easy mechanism currently in Linux to avoid this.
Why is this a problem? The portable version of OpenSSH has had its share of remotely exploitable bugs and through these bugs attackers have gained root access into the target systems. This problem is not specific to OpenSSH, but common to all network processes running with root access.
Since OpenSSH server will announce its version to whomever that connects to the port, finding a suitable exploit program for known exploitable version of the service is not hard. The announcement feature is part of the SSH-protocol so it cannot be changed.
We're left with some possibilities then:
- Do not run SSH server (makes administration over networks difficult at best).
- Replace SSH with something else (but what?). Sometimes the physical hardware or the virtualization environment provides some access mechanism to the system. Remote control cards/chips are fairly common nowadays in proper server hardware so this possibility could be explored. Allowing access to software running on the remote control card might not be such a good idea either, so stop and think before you enable that service.
- Run SSH on a non-default port (but the banner is still seen by anyone who does a full port scan).
- Firewall SSH so that it only accepts connections coming from a known IP address range (an idea that actually works, but cannot be always realized). If you run multiple servers in the same network, it might be wise to arrange one host that accepts SSH from the external world and use that host to connect to all the servers. This way the SSH daemons on the servers are not exposed directly to Internet. This kind of gateway server is often called an "SSH gateway".
- Require IPSec transport first. This moves the security problem into IPSec-implementation and since IPSec has its own security risks, we won't be covering it here.
- Use a technique called "port knocking". Use google to find out about this. Makes attacking the service much harder if implemented properly. Not covered here.
- Live with the risk but attempt to migitate it. This is what we'll do here.
SSH server provides some security mechanisms that are related to the application layer. Since it normally is built with TCP Wrappers, it does not have its own IP or DNS-based access control lists. However, we'll want to change the default configuration so that we reduce the attack vectors into SSH.
We want to:
- Disable root login even if the attacker guesses the root password. By default root login is enabled.
- Enable logging in through SSH only for users in a specific group. We'll create a special purpose group (sshers) for this purpose. This means that by default users cannot login through SSH even if they can connect to it and know their password.
- Disable all extra features from SSH which are not relevant for an web server. Extra features mean more executed code which in turn means more attack vectors.
- Disable protocol version 1. The first version of the protocol had a design flaw by which it is possible to exploit existing SSHv1 connections in a local network. Since we don't know the number or locations of all local networks through which the connection will travel, we just disable the first version support. All publicly available SSH clients support version 2 so this won't be a problem.
In the process we'll create an user account for testing which will be used to verify that only users in the sshers group can login. We'll remove the account at the end. We also test that port forwarding doesn't work anymore (by connecting to a bogus remote port, in order to generate an error message).
Note that the new contents of the configuration file follows the command listing.
[user@centos ~]$ cd /etc/ssh/
[user@centos ssh]$ ls -al
total 200
drwxr-xr-x 2 root root 4096 Oct 16 19:17 .
drwxr-xr-x 74 root root 4096 Oct 16 20:43 ..
-rw------- 1 root root 111892 Aug 13 01:48 moduli
-rw-r--r-- 1 root root 1417 Aug 13 01:48 ssh_config
-rw------- 1 root root 3025 Aug 13 01:48 sshd_config
-rw------- 1 root root 668 Oct 16 19:17 ssh_host_dsa_key
-rw-r--r-- 1 root root 590 Oct 16 19:17 ssh_host_dsa_key.pub
-rw------- 1 root root 515 Oct 16 19:17 ssh_host_key
-rw-r--r-- 1 root root 319 Oct 16 19:17 ssh_host_key.pub
-rw------- 1 root root 887 Oct 16 19:17 ssh_host_rsa_key
-rw-r--r-- 1 root root 210 Oct 16 19:17 ssh_host_rsa_key.pub
[user@centos ssh]$ sudo vi sshd_config
[user@centos ssh]$ sudo /usr/sbin/useradd tester
[user@centos ssh]$ sudo passwd tester
Changing password for user tester.
New UNIX password: Enter password for the tester-account
BAD PASSWORD: it does not contain enough DIFFERENT characters
Retype new UNIX password: Retype the password. Even if it
is a bad one, it will be accepted since we're executing
using root-privileges
passwd: all authentication tokens updated successfully.
[user@centos ssh]$ sudo /usr/sbin/groupadd sshers
[user@centos ssh]$ sudo /usr/sbin/usermod -a -G sshers user
[user@centos ssh]$ id user
uid=500(user) gid=500(user) groups=500(user),10(wheel),502(sshers)
[user@centos ssh]$ sudo /etc/init.d/sshd restart
Stopping sshd: [ OK ]
Starting sshd: [ OK ]
[user@centos ssh]$ sudo /sbin/service sshd restart
Stopping sshd: [ OK ]
Starting sshd: [ OK ]
[user@centos ssh]$ sudo su - tester
[tester@centos ~]$ ssh localhost
The authenticity of host 'localhost (127.0.0.1)' can't be established.
RSA key fingerprint is d5:92:f9:82:75:50:73:a7:f2:80:8b:90:2d:1e:fc:e5.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
tester@localhost's password: Enter tester's password
Permission denied, please try again.
tester@localhost's password: Enter it again
Permission denied, please try again.
tester@localhost's password: Doesn't seem to work. The password is correct though
Permission denied (publickey,password).
[tester@centos ~]$ exit
[user@centos ssh]$ ssh localhost
The authenticity of host 'localhost (127.0.0.1)' can't be established.
RSA key fingerprint is d5:92:f9:82:75:50:73:a7:f2:80:8b:90:2d:1e:fc:e5.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'localhost' (RSA) to the list of known hosts.
user@localhost's password: Enter your normal account password
Last login: Mon Oct 16 20:44:30 2006 from itchy.intranet
[user@centos ~]$ sudo tail /var/log/secure
Password: Normal account password. Since we started a new
session, the system doesn't remember our previous sudo-authentication.
Oct 16 21:14:49 centos sshd[3809]: Failed password for invalid user tester from ::ffff:127.0.0.1 port 32769 ssh2
Oct 16 18:14:49 centos sshd[3810]: Failed password for invalid user tester from ::ffff:127.0.0.1 port 32769 ssh2
Oct 16 21:14:54 centos sshd[3809]: Failed password for invalid user tester from ::ffff:127.0.0.1 port 32769 ssh2
Oct 16 18:14:54 centos sshd[3810]: Failed password for invalid user tester from ::ffff:127.0.0.1 port 32769 ssh2
Oct 16 21:14:58 centos sshd[3809]: Failed password for invalid user tester from ::ffff:127.0.0.1 port 32769 ssh2
Oct 16 18:14:58 centos sshd[3810]: Failed password for invalid user tester from ::ffff:127.0.0.1 port 32769 ssh2
Oct 16 18:14:58 centos sshd[3810]: Connection closed by ::ffff:127.0.0.1
Oct 16 18:15:35 centos sshd[3841]: Accepted password for user from ::ffff:127.0.0.1 port 32770 ssh2
Oct 16 21:15:35 centos sshd[3840]: Accepted password for user from ::ffff:127.0.0.1 port 32770 ssh2
[user@centos ~]$ ssh -1 localhost
Protocol major versions differ: 1 vs. 2
[user@centos ~]$ sudo tail -n 2 /var/log/secure
Oct 16 21:18:20 centos sshd[3901]: Did not receive identification string from ::ffff:127.0.0.1
Oct 16 21:18:23 centos sudo: user : TTY=pts/1 ; PWD=/home/user ; USER=root ; COMMAND=/usr/bin/tail -n 2 /var/log/secure
[user@centos ~]$ ssh -f -L 1234:localhost:80 localhost sleep 30
user@localhost's password: Enter normal account password
[user@centos ~]$ telnet localhost 1234
Trying 127.0.0.1...
channel 3: open failed: administratively prohibited: open failed
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
Connection closed by foreign host.
[user@centos ~]$ sudo tail -2 /var/log/secure
Password: Enter normal password
Oct 16 21:28:27 centos sshd[4020]: Received request to connect to host localhost port 80, but the request was denied.
Oct 16 21:28:46 centos sudo: user : TTY=pts/1 ; PWD=/home/user ; USER=root ; COMMAND=/usr/bin/tail -2 /var/log/secure
[user@centos ~]$ sudo /usr/sbin/userdel -r tester
[user@centos ~]$ id tester
id: tester: No such user
[user@centos ~]$ ls -la /home/tester
ls: /home/tester: No such file or directory
[ Securing SSH and testing ]
Two ways to restart a service:
- /etc/init.d/service-script command : This is the more portable way since it will work on other distributions as well.
- /sbin/service service-script command : This one is preferred by Red Hat documentation, but internally calls the above service script anyway.
# $OpenBSD: sshd_config,v 1.69 2004/05/23 23:59:53 dtucker Exp $
# This is the sshd server system-wide configuration file. See
# sshd_config(5) for more information.
# This sshd was compiled with PATH=/usr/local/bin:/bin:/usr/bin
# The strategy used for options in the default sshd_config shipped with
# OpenSSH is to specify options with their default value where
# possible, but leave them commented. Uncommented options change a
# default value.
#Port 22
# !! NOTE. Modified to accept only version 2 (default is 2,1)
Protocol 2
#ListenAddress 0.0.0.0
#ListenAddress ::
# HostKey for protocol version 1
#HostKey /etc/ssh/ssh_host_key
# HostKeys for protocol version 2
#HostKey /etc/ssh/ssh_host_rsa_key
#HostKey /etc/ssh/ssh_host_dsa_key
# Lifetime and size of ephemeral version 1 server key
#KeyRegenerationInterval 1h
#ServerKeyBits 768
# Logging
#obsoletes QuietMode and FascistLogging
#SyslogFacility AUTH
SyslogFacility AUTHPRIV
#LogLevel INFO
# Authentication:
#LoginGraceTime 2m
# !! NOTE modified to not allow root logins even if password is correct
PermitRootLogin no
#StrictModes yes
#MaxAuthTries 6
#RSAAuthentication yes
#PubkeyAuthentication yes
#AuthorizedKeysFile .ssh/authorized_keys
# For this to work you will also need host keys in /etc/ssh/ssh_known_hosts
#RhostsRSAAuthentication no
# similar for protocol version 2
#HostbasedAuthentication no
# Change to yes if you don't trust ~/.ssh/known_hosts for
# RhostsRSAAuthentication and HostbasedAuthentication
#IgnoreUserKnownHosts no
# Don't read the user's ~/.rhosts and ~/.shosts files
#IgnoreRhosts yes
# To disable tunneled clear text passwords, change to no here!
#PasswordAuthentication yes
# !! NOTE verify that this is "no".
#default in modern OpenSSH servers is "no".
#PermitEmptyPasswords no
PasswordAuthentication yes
# Change to no to disable s/key passwords
#ChallengeResponseAuthentication yes
ChallengeResponseAuthentication no
# Kerberos options
# !! NOTE do not enable kerberos unless you know what you're doing
#KerberosAuthentication no
#KerberosOrLocalPasswd yes
#KerberosTicketCleanup yes
#KerberosGetAFSToken no
# GSSAPI options
#GSSAPIAuthentication no
# !! NOTE disabled GSSAPI-based auth. Not relevant unless
# you run kerberos
GSSAPIAuthentication no
#GSSAPICleanupCredentials yes
GSSAPICleanupCredentials yes
# Set this to 'yes' to enable PAM authentication, account processing,
# and session processing. If this is enabled, PAM authentication will
# be allowed through the ChallengeResponseAuthentication mechanism.
# Depending on your PAM configuration, this may bypass the setting of
# PasswordAuthentication, PermitEmptyPasswords, and
# "PermitRootLogin without-password". If you just want the PAM account and
# session checks to run without PAM authentication, then enable this but set
# ChallengeResponseAuthentication=no
#UsePAM no
UsePAM yes
# !! NOTE Disabled port forwarding
AllowTcpForwarding no
#GatewayPorts no
#X11Forwarding no
# !! NOTE Also disabled X11 forwarding (just in case)
X11Forwarding no
#X11DisplayOffset 10
#X11UseLocalhost yes
#PrintMotd yes
#PrintLastLog yes
#TCPKeepAlive yes
#UseLogin no
#UsePrivilegeSeparation yes
#PermitUserEnvironment no
#Compression yes
#ClientAliveInterval 0
#ClientAliveCountMax 3
# !! NOTE in broken DNS-environments where PTR -> A -> PTR
# lookups fail you should probably disable the
# following option.
#UseDNS yes
#PidFile /var/run/sshd.pid
#MaxStartups 10
#ShowPatchLevel no
# no default banner path
#Banner /some/path
# override default of no subsystems
# !! NOTE disabled sftp support by commenting the following
# line
# Subsystem sftp /usr/libexec/openssh/sftp-server
# !! NOTE this is new:
# we'll create a group called 'sshers' to control
# which users can login over ssh.
# by using AllowGroups we'll implicitly disable logins
# from all other users
# Other user-specific support is available by using
# the following sshd configuration words:
# - AllowUsers
# - DenyUsers & DenyGroups
AllowGroups sshers
[ The new SSH daemon configuration file ]
A lot of network services were developed originally without concepts of network security. Since adding network security mechanisms into existing services was deemed too painful, Wietse Venema started working on a simple library that would handle all the network related access control aspects while providing a simple interface for the network service.
TCP Wrappers works so that a network process will call a routine from the library that will wait for new incoming connections. When a connection eventually arrives on the network socket, the library will consult two files (/etc/hosts.allow and /etc/hosts.deny) to determine whether the connection should be allowed or dropped. If it is dropped, the routine will generate a log message and continue waiting for the next incoming connection. In this way the network service can concentrate on providing the service and not worry about access control mechanisms related to network.
Since TCP Wrappers only reads two configuration files for all services, the configuration files normally contain names of services to which each access control is related to.
Steps that TCP Wrappers library uses to decide whether a connection is allowed:
- At first, it will need a service name to work with. Normally a service name is the zeroth command line argument (being the program/process name).
- Combined with this service name and the remote connection address (IP at this point), it will proceed to look for rules in the configuration files on whether this connection is allowed or not.
- Next the wrapper library needs to find out whether to allow the connection.
- It will consult /etc/hosts.allow in order to see whether the service name / connection address pair matches any entries there. The first entry to match will allow the connection to proceed. If there are no matches, the wrapper library will advance to the next step.
- Next it will search for a match in /etc/hosts.deny. The syntax for this file is the same as the previous one, but if it can find a match in this file, the connection will be denied (hence the filename). If there's no match in the /etc/hosts.deny either, the wrapper will proceed to next step.
- The connection didn't match in either file, so it is accepted. This might strike a bit odd at first. TCP Wrappers was meant to be a drop-in library and in this case one could start with empty configuration files and nothing would break.
To reiterate, if the incoming connection first matches in the allow file, it is allowed. If it didn't match in allow but matches in deny, it will denied. If it doesn't match in either file, it will be accepted.
The syntax of the rules for wrappers is as follows:
- Each rule is presented on a single line of text
- The rule is split using a colon into two parts:
- A comma separated list of service identifiers (service name or ANY)
- A comma separated list of client identifiers:
- IP-address (172.16.1.2)
- DNS-name (will cause reverse DNS lookup) (mycomp.example.com).
Please note that whenever using DNS names for access control, you're trusting your DNS infrastructure. There are a lot of techniques to use against DNS (poisoning, building shadow forward-zones, etc) so using DNS for network security access control is NOT RECOMMENDED.
- A pattern of IP or DNS names (.domain.com, 192.168. )
- An IP-address / netmask combination (10.2.0.0/255.255.0.0, CIDR is not supported for IPv4 addresses)
- A separate file containing the patterns (similar to "include" in many other configuration languages)
- Executing a program with parameters that can decide whether the connection can proceed.
- And other more complicated choices which you can read about by using man hosts.allow .
Let's start with a simple setup for our system.
A sane collection of rules for any system would:
- By default drop all connections which are not explicitly allowed.
- Allow connections originating from our own system (otherwise a lot of seemingly simple things break).
- Allow connection to specific network services only from specific clients (if possible).
Based on the first criteria, we already know that our /etc/hosts.deny will contain one line. Instead of listing all the services, we use the special keyword ALL. The same keyword is also available for client list.
# match all services and all clients
# since this is the .deny file, all connections not matching in
# .allow file will be denied.
ALL: ALL
[ By default we'll drop all connections ]
We then have to implement the next two rules. We'll start by allowing connections to ALL TCP Wrappers protected services originating from localhost and then allow connections to our sshd service from everywhere on the Internet.
# This file lists the allowed connections
# We allow connecting to all wrapper protected services
# if the connection seems to originate from localhost
# In order to make the match faster, we'll use the
# IP address of localhost (to save one DNS roundtrip)
ALL: 127.0.0.1
# We also allow connecting to our sshd process from all
# clients
sshd: ALL
[ We allow only these connections ]
When modifying the configuration files, you don't need to restart services. Each time a connection arrives, the library will check whether the configuration files have been changed and reload them automatically. This is different from most service-level configuration files with which you need to service to reload the configuration (or restart the service).
Testing whether the wrapper configuration works can be done using same techniques as testing for the echo service below (in the section about xinetd).
At this point you should ask "how do I know whether a service can be protected using TCP wrappers?". Since not all network services use the wrapper library, it is useful to know how to determine if a service uses the library or not. For most network services, this is a build-time option that has been selected for you by your Linux distributor (in our case Red Hat, since Centos uses the same build-settings).
Let's start with an example in which we determine that our ssh service uses wrappers and the above configuration actually protects it.
[user@centos ~]$ which sshd
/usr/bin/which: no sshd in (/usr/kerberos/bin:/usr/local/bin:/bin:/usr/bin:/usr/X11R6/bin:/home/user/bin)
[user@centos ~]$ rpm -q sshd
package sshd is not installed
[user@centos ~]$ rpm -qa | grep ssh
openssh-3.9p1-8.RHEL4.15
openssh-clients-3.9p1-8.RHEL4.15
openssh-server-3.9p1-8.RHEL4.15
openssh-askpass-gnome-3.9p1-8.RHEL4.15
openssh-askpass-3.9p1-8.RHEL4.15
[user@centos ~]$ rpm -ql openssh-server | grep sshd
/etc/pam.d/sshd
/etc/rc.d/init.d/sshd
/etc/ssh/sshd_config
/usr/sbin/sshd
/usr/share/man/man5/sshd_config.5.gz
/usr/share/man/man8/sshd.8.gz
/var/empty/sshd
[user@centos ~]$ ldd /usr/sbin/sshd
libwrap.so.0 => /usr/lib/libwrap.so.0 (0x00813000)
libpam.so.0 => /lib/libpam.so.0 (0x008e7000)
libdl.so.2 => /lib/libdl.so.2 (0x00597000)
libaudit.so.0 => /lib/libaudit.so.0 (0x0024a000)
libcrypto.so.4 => /lib/libcrypto.so.4 (0x00e02000)
libutil.so.1 => /lib/libutil.so.1 (0x00592000)
libz.so.1 => /usr/lib/libz.so.1 (0x00be6000)
libnsl.so.1 => /lib/libnsl.so.1 (0x00f10000)
libcrypt.so.1 => /lib/libcrypt.so.1 (0x00311000)
libselinux.so.1 => /lib/libselinux.so.1 (0x00151000)
libgssapi_krb5.so.2 => /usr/lib/libgssapi_krb5.so.2 (0x001a5000)
libkrb5.so.3 => /usr/lib/libkrb5.so.3 (0x001b9000)
libk5crypto.so.3 => /usr/lib/libk5crypto.so.3 (0x009d8000)
libcom_err.so.2 => /lib/libcom_err.so.2 (0x00111000)
libresolv.so.2 => /lib/libresolv.so.2 (0x00114000)
libc.so.6 => /lib/tls/libc.so.6 (0x0033f000)
/lib/ld-linux.so.2 (0x00c1a000)
[user@centos ~]$ rpm -qif /usr/lib/libwrap.so.0
Name : tcp_wrappers Relocations: (not relocatable)
Version : 7.6 Vendor: CentOS
Release : 37.2 Build Date: Mon 21 Feb 2005 05:53:17 PM EET
Install Date: Mon 16 Oct 2006 07:13:21 PM EEST Build Host: bhrama.build.karan.org
Group : System Environment/Daemons Source RPM: tcp_wrappers-7.6-37.2.src.rpm
Size : 227955 License: Distributable
Signature : DSA/SHA1, Sat 26 Feb 2005 11:39:33 PM EET, Key ID a53d0bab443e1821
Packager : Karanbir Singh <kbsingh@centos.org>
URL : ftp://ftp.porcupine.org/pub/security/index.html
Summary : A security tool which acts as a wrapper for TCP daemons.
Description :
The tcp_wrappers package provides small daemon programs which can
monitor and filter incoming requests for systat, finger, FTP, telnet,
rlogin, rsh, exec, tftp, talk and other network services.
Install the tcp_wrappers program if you need a security tool for
filtering incoming network services requests.
This version also supports IPv6.
[user@centos ~]$ rpm -qld tcp_wrappers
/usr/share/doc/tcp_wrappers-7.6/BLURB
/usr/share/doc/tcp_wrappers-7.6/Banners.Makefile
/usr/share/doc/tcp_wrappers-7.6/CHANGES
/usr/share/doc/tcp_wrappers-7.6/DISCLAIMER
/usr/share/doc/tcp_wrappers-7.6/README
/usr/share/doc/tcp_wrappers-7.6/README.IRIX
/usr/share/doc/tcp_wrappers-7.6/README.NIS
/usr/share/man/man3/hosts_access.3.gz
/usr/share/man/man5/hosts.allow.5.gz
/usr/share/man/man5/hosts.deny.5.gz
/usr/share/man/man5/hosts_access.5.gz
/usr/share/man/man5/hosts_options.5.gz
/usr/share/man/man8/tcpd.8.gz
[ Determining whether a service can be protected with TCP Wrappers ]
You can use the same technique to find out whether a program uses some library. It will work as long as the library has been linked dynamically. Statically linked libraries require extra trickery (there is no common solution).
What if you want a list of all programs that are linked against libwrap.so? This will require either doing a lot of manual labor, or the following power command. It is quite heavy on the resources, so think twice before running it. And yes, it's optional.
[user@centos ~]$ find / -perm -111 -type f \! -name "*.so*" -exec sh -c \
'file -i {} | grep -q application/x-sharedlib && ldd {} | grep -q libwrap && echo {}' \; \
2> /dev/null
/usr/bin/rmail.sendmail
/usr/bin/gdm-binary
/usr/sbin/praliases
/usr/sbin/xinetd
/usr/sbin/sshd
/usr/sbin/makemap
/usr/sbin/rpc.mountd
/usr/sbin/mailstats
/usr/sbin/smrsh
/usr/sbin/rpc.rquotad
/usr/sbin/sendmail.sendmail
/sbin/rpc.statd
[user@centos ~]$ ldd /sbin/portmap
libnsl.so.1 => /lib/libnsl.so.1 (0x001c8000)
libc.so.6 => /lib/tls/libc.so.6 (0x00693000)
/lib/ld-linux.so.2 (0x00b87000)
[ Finding all executables that use a dynamically linked library ]
Decoding the power command is left as an additional exercise to the reader (use the man-pages Luke!). You might also notice that some programs do not show up in the list even if they use the wrapper. On Centos/RHEL the portmap program responsible in establishing service end-points for RPC programs (like NFS and such) links against the wrapper library statically. Try to find out how to establish this (hint: strings and grep).
xinetd is a modern implementation of an internet super server (no, it doesn't make your Internet more super). Long time ago (in a galaxy far away) network services were simple. In fact, they were so simple that they only processed one instruction at a time from one client and then terminated. Computers back then didn't have a lot of RAM (certainly less than 64 MiB). In this kind of environment it made sense to start network server code only when a client actually requested service. This is a different model from modern network servers which start once and then wait for a client to connect at some later point (if ever). The basic idea behind the internet super server is that it will listen on predefined network ports and once it will get a connection from a client, it will launch a program to handle that one client.
The original version of the super server is called inetd. It had fairly limited configuration facilities and had only one configuration file in which all the ports and the programs to launch were listed. It did not have it's own network security settings but provided very primitive connection rate throttling. The original inetd is still used in some distributions, but Centos/RHEL have moved to use a more advanced version which is better suited for modern environments. This version is called xinetd but still solves the same problem.
There are still couple of cases when you might want to actually run xinetd. You want to provide telnet service to users (bad idea since SSH exists) or you want to run a TFTP-server (mainly used for booting computers and devices over the network). In any case, it is useful to at least know how the configuration process works.
The service configuration has been split into multiple files in Centos/RHEL. The main file /etc/xinetd.conf contains an include-directive that will automatically read all the files under /etc/xinetd.d/ and pretend that those directives were present in the main file. This way if you need to enable a service, you just need to find its relevant xinetd configuration file under /etc/xinetd.d/, modify it, and then reload xinetd.
It is also possible to use chkconfig to enable/disable services behind xinetd but the xinetd service needs to be enabled first.
We'll enable the echo-service in order to test out how to make more complicated TCP Wrappers rules. xinetd is linked against the wrapper library so we can protect the services provided by xinetd using the files that you've already seen before.
The service name to use in /etc/hosts.{allow,deny} is the name given in the id field of the xinetd configuration.
# default: off
# description: An xinetd internal service which echo's characters back to clients. \
# This is the tcp version.
service echo
{
type = INTERNAL
id = echo-stream
socket_type = stream
protocol = tcp
user = root
wait = no
# enable for testing (enable == disable=no)
disable = no
}
[ Contents of the echo -configuration file ]
In order to enable an service inside xinetd you need to un-disable it (which has been done above). After this you need to tell xinetd to reread the configuration or start it (it was stopped by us before).
We next need to setup the TCP Wrappers settings for the echo service:
# Append the following fragment to the end of your hosts.allow
# the following is an internal service in xinetd
# in which case the xinetd-conf 'id' field is the name
# we only allow host blart.intranet to use this service
# (localhost testing is still possible)
echo-stream: blart.intranet
[ Addition into /etc/hosts.allow ]
And finally we get to test the wonderfully simple service (with telnet). We first connect from localhost (so it should work) and next connect by using an IP that is the primary IP of one of our network cards. Since we no longer use 127.0.0.1 and our host is not blart.intranet, wrappers will deny access to us and drop the connection.
[user@centos ~]$ sudo vi /etc/xinetd.d/echo
[user@centos ~]$ /sbin/chkconfig --list xinetd
xinetd 0:off 1:off 2:off 3:off 4:off 5:off 6:off
[user@centos ~]$ sudo /sbin/service xinetd start
Starting xinetd: [ OK ]
[user@centos ~]$ sudo netstat -tlp
Password:
Active Internet connections (only servers)
Proto Local Address Foreign Address PID/Program name
tcp *:echo *:* 21288/xinetd
tcp localhost.localdomain:smtp *:* 2339/sendmail: acce
tcp *:http *:* 3579/httpd
tcp *:ssh *:* 3777/sshd
tcp *:https *:* 3579/httpd
[user@centos ~]$ telnet localhost echo
Trying 127.0.0.1...
Connected to localhost.localdomain (127.0.0.1).
Escape character is '^]'.
Hello
Hello
<Ctrl+AltGr+]>close
telnet> close
Connection closed.
[user@centos ~]$ /sbin/ifconfig eth0 | grep inet
inet addr:192.168.1.30 Bcast:192.168.1.255 Mask:255.255.255.0
[user@centos ~]$ telnet 192.168.1.30 echo
Trying 192.168.1.30...
Connected to centos.intranet (192.168.1.30).
Escape character is '^]'.
Connection closed by foreign host.
[user@centos ~]$ sudo tail -3 /var/log/secure
Oct 19 20:17:38 centos xinetd[3076]: START: echo-stream pid=3120 from=192.168.1.30
Oct 19 20:17:38 centos xinetd[3120]: FAIL: echo-stream libwrap from=192.168.1.30
Oct 19 20:17:48 centos sudo: user : TTY=pts/0 ; PWD=/home/user ; USER=root ; \
COMMAND=/usr/bin/tail -3 /var/log/secure
[ Testing xinetd and TCP Wrappers ]
And finally we arrive at our last stop on the grand tour of network security mechanisms: the kernel firewall code (also known as netfilter).
When all else fails, you can always apply network security policy using the built-in firewall. In most cases you should use it even if you have higher-level mechanisms in place since it will just add to the available security. Since we're still securing a web server, you might have noticed that Apache web server wasn't ever seen in the TCP Wrappers lists. This is because Apache doesn't use TCP Wrappers. It does have rather rich ACL language available internally so TCP Wrappers do not really provide anything that is not already possible within Apache.
Since netfilter works inside the kernel, it has a fairly limited view of the world. For example, you cannot use DNS to limit connections. On the other hand, you would not use DNS for limiting anyway since it is prone to external spoofing.
The first Linux kernel firewall was introduced in version 2.0 (about ten years ago) and until version 2.4 the firewalling code was stateless. This meant that each incoming or outgoing network packet was analyzed by itself and decision on whether it could pass had to be done based on the packet contents only (no other information). Starting from the kernel version 2.4, the firewall was redone completely, and the basic architecture has stayed the same even in 2.6 series. There is still some unification going on (like unifying the IPv4 and IPv6 code) but other than that the basic model is the same. The system is modular (can be extended with more functionality) and very flexible. It also can be stateful (via the state module) and do packet inspection at higher levels than just L2 or L3 (depending on the module in question).
We will cover netfilter at a simple level so that we don't concern ourselves with routing or network address translation (NAT).
Before we dive into netfilter and the iptables tool that is used to program the netfilter rules, some terminology and structure needs to be defined.
The simplified model of netfilter consists of hierarchical elements.
The element hierarchy is:
- The highest-level element is a Table. A table is related to some specific function of the netfilter system. We will only be using the filter table, and it is also the default table that iptables manipulates. The other tables are related to NAT-functions, L2-transparent manipulation and packet rewriting in general.
- Each table contains a fixed set of Chains each of which will be consulted at a different phase of packet transfer. The three chains that already exist in filter table are:
- OUTPUT: consists of list of rules to apply for each packet generated by processes running on our system (i.e. traffic generated by our network services). This chain cannot be removed (although it can be empty).
- INPUT: consists of list of rules to apply for each packet received by our system when the packet is travelling toward some process on our system. This chain cannot be removed.
- FORWARD: list of rules to use for packets which our system receives but are not destined to our system. This chain would be consulted for routed packets. Since we will not be running routing on our web server, we will just DROP all packets that hit this chain (see below). This chain cannot be removed.
- A large possible number of user specified additional chains. These can be used from the three above chains if necessary but we keep things simple and not use any. Each user specified chain will have some unique name by which it can be used by rules in the pre-defined or other user specified chains. User-specified chains can be removed.
- A chain is a list of 0 to N rules:
- Each rule consists of a MATCH SPECIFICATION and a TARGET if the match specification matches a packet. There are many targets and we can also extend them by using new netfilter modules, but again, for simplicity, we'll concentrate on the simple ones.
- The built-in chains (INPUT, OUTPUT, and FORWARD) also
have a default POLICY. This is a TARGET which will be used for all packets that do not match any of the rules in the chain.
Let's take a simple command example:
iptables -A INPUT -p tcp --dport 22 -j ACCEPT
[ Adding a new rule to INPUT-chain. ]
The match specification is in bold, the target is in italic.
Let's dissect the command:
- -A INPUT : Append a new rule into chain called INPUT. Note that netfilter like 99.9% of UNIX and Linux is case-sensitive. The built-in rules always use all capital letters.
- -p tcp --dport 22 : Our match specification consists of two parts which both must match.
In general, all the different parts that make up a match specification must all match. Netfilter does not support complex boolean matches directly. If you want more complex rules you'll have to use more than one rule.
- -p tcp : Protocol field (/etc/protocols) in IP packet must be equal to 6 (number reserved for TCP packets on IP). As you can see, iptables can use the /etc/protocols file directly so that you don't have to remember the protocol numbers. You could also use 6 .
- --dport 22 : The destination port field within the TCP header must be equal to 22 (SSH). Since other protocols also have "port numbers", it is important to first specify the lower layer fields and then the higher layer fields. In this case --dport must always be preceded by -p, otherwise your rule will not work.
- -j ACCEPT : Targets are specified by the Jump-"option". The destination for a jump can be a built-in target (like ACCEPT) or a name for a user specified chain.
Netfilter comes with three targets built in:
- ACCEPT : let the packet proceed to the next phase of packet transfer inside the kernel. In our case this will either be the network interface card, or our process.
- DROP : forget about this packet. No other processing will be done on it.
- REJECT : similar to DROP, but also send an ICMP error message to the IP source address of the packet to notify them that the packet was dropped. This is mainly useful when you're dropping traffic that was generated within your LAN or a remote network that you trust. This will make connection failures much faster since the remote host will know that connection is not possible and will not assume that the connection packet was just lost and resend it.
Other targets can be loaded (we'll use the LOG target later on) but these three are the most important ones.
We are now armed and ready to dissect the "firewall" rules that the install program generated for us. We will first check out the tool that Red Hat provides for us for this. The tool was originally called lokkit (and still will start with that name as well) but the more fashionable name is system-config-securitylevel-tui. There is also system-config-securitylevel which is the graphical version of the tool and it also allows us set modify some aspects of SELinux. We'll use the graphical version later on.
[user@centos sysconfig]$ sudo system-config-securitylevel-tui
[ Starting lokkit, err, system-config-securitylevel-tui ]
[ Lokkit main screen ]
We then select "Customize" assuming that we can do something with this tool.
[ Lokkit "customization" screen ]
As you can see there not a lot that you can do with lokkit. The trusted interface option should only be used when you have your system connected to multiple computers and you absolutely trust a network behind one network card. This is normally not the case since you do not control all the systems on the "trusted network" so it is rarely a good idea to enable this.
Do not modify anything. Just select "Ok" and leave the tool.
What lokkit fails to communicate is that it doesn't really show you the whole truth. In fact, there are a lot of ports open which are not shown in lokkit at all. This is why we'll now forget about lokkit and promise never to use it again.
Instead, we'll be using the iptables command directly and start by analyzing the current firewall settings. After that we'll proceed to make our own (and more secure) set of rules.
[user@centos ~]$ /sbin/iptables -vnL
iptables v1.2.11: can't initialize iptables table `filter': Permission denied (you must be root)
Perhaps iptables or your kernel needs to be upgraded.
[user@centos ~]$ sudo /sbin/iptables -vnL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
47441 4275K RH-Firewall-1-INPUT all -- * * 0.0.0.0/0 0.0.0.0/0
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
0 0 RH-Firewall-1-INPUT all -- * * 0.0.0.0/0 0.0.0.0/0
Chain OUTPUT (policy ACCEPT 40832 packets, 4405K bytes)
pkts bytes target prot opt in out source destination
Chain RH-Firewall-1-INPUT (2 references)
pkts bytes target prot opt in out source destination
26774 2577K ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 icmp type 255
0 0 ACCEPT esp -- * * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT ah -- * * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT udp -- * * 0.0.0.0/0 224.0.0.251 udp dpt:5353
0 0 ACCEPT udp -- * * 0.0.0.0/0 0.0.0.0/0 udp dpt:631
20489 1657K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
16 1000 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:22
0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 state NEW tcp dpt:80
162 39690 REJECT all -- * * 0.0.0.0/0 0.0.0.0/0 reject-with icmp-host-prohibited
[user@centos ~]$ sudo cat /etc/sysconfig/iptables
# Firewall configuration written by system-config-securitylevel
# Manual customization of this file is not recommended.
*filter
:INPUT ACCEPT [0:0]
:FORWARD ACCEPT [0:0]
:OUTPUT ACCEPT [0:0]
:RH-Firewall-1-INPUT - [0:0]
-A INPUT -j RH-Firewall-1-INPUT
-A FORWARD -j RH-Firewall-1-INPUT
-A RH-Firewall-1-INPUT -i lo -j ACCEPT
-A RH-Firewall-1-INPUT -p icmp --icmp-type any -j ACCEPT
-A RH-Firewall-1-INPUT -p 50 -j ACCEPT
-A RH-Firewall-1-INPUT -p 51 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp --dport 5353 -d 224.0.0.251 -j ACCEPT
-A RH-Firewall-1-INPUT -p udp -m udp --dport 631 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 22 -j ACCEPT
-A RH-Firewall-1-INPUT -m state --state NEW -m tcp -p tcp --dport 80 -j ACCEPT
-A RH-Firewall-1-INPUT -j REJECT --reject-with icmp-host-prohibited
COMMIT
[user@centos ~]$ getent protocols 50
ipv6-crypt 50 IPv6-Crypt
[user@centos ~]$ getent protocols 51
ipv6-auth 51 IPv6-Auth
[user@centos ~]$ getent services 5353
[user@centos ~]$ getent services 631
ipp 631/tcp
[ Inspecting current firewalling setup ]
Short explanations for the commands and files:
- iptables -vnL : List rules. Be Verbose. Use Numbers instead of symbolic names.
Since we didn't specify a chain name, all chains will be listed. Since we didn't specify the table (-t filter) by default the filter table will be used. The main use for this form of output are the statistics counters. Netfilter keeps counts (both packet and bytes) for each rule that matches. Counts are also kept for default policy targets (as you can see in the OUTPUT chain).
- /etc/sysconfig/iptables : This is the file that contains all the rules in a format suitable to be loaded into iptables directly (we'll cover this later).
Reading this file directly is easier in many cases as the verbose listing doesn't always show all the match specification parts. We first see the table name (filter), and then we see the chain definitions (they start with a colon). The chain definitions must always have their default policy listed (some built-in TARGET).
After the chain definitions each line is a command to create a new rule by appending to each chain. You will notice that the Red Hat way of doing things is to define a new custom chain which will be used for both incoming and forwarded (routed) packets. We will do away with such trickery in our rules.
- getent protocols 50 : Do a lookup into the protocol database in the system looking for a protocol with the number 50.
You could also read the contents of /etc/protocols file (which is the protocol database). In this case we're interested what the -p 50 match specification might be related to. IP protocol 50 is ESP (IPSec encrypted and authenticated payload).
- Protocol 51 is AH (IPSec authenticated payload).
- getent services 5353 : We try to find which service has been allocated the port 5353 (in UDP) by IANA.
It seems that IANA has not allocated this port number to anyone. So after using google we find out that this is some kind of multicast-DNS resolution service. The destination (-d 244.0.0.251) address is a multicast address, providing us another hint on the nature of this service.
- Port 631 has been allocated to IPP (Internet Printing Protocol) at least on the TCP-side of things.
We assume that this is some UDP version of broadcasting used to locate printers. Google would help. Please note that if you connect to 631/TCP with your browser, you could then access the web interface of CUPS administration. The same port is also used to submit printing jobs and to support other printing functions. We don't have CUPS running on our system, so this doesn't apply to our system.
Even if the original ruleset file says that manual modification is not recommended, we'll modify it anyway (we could also rewrite the whole file from scratch). The reason for the warning is that if you modify the ruleset manually, the changes will be lost if you re-run lokkit again. Since we will not be using lokkit to modify the new ruleset, this won't be a problem for us.
As for the other bits and pieces that you might not be familiar with (yet):
- -i lo : matches packets which have been received from given interface. In this case, lo which is the interface name for the loopback adapter (by default, handles the 127/8 network which includes our localhost address). We allow all packets coming from the loopback address. This is a good idea.
- -p icmp --icmp-type any : Could be abbreviated as -p icmp since we don't care which ICMP message codes are present.
- -p udp -m udp --dport 631 : -m udp means "load a module called udp to extend functionality present in iptables". Since the module is always present in iptables, this match part is unnecessary.
-p udp --dport 631 then will match an IPv4 packet which is carrying UDP payload and the UDP destination port must be 631.
- -m state : This is where things get interesting. We ask the tool to load the state module which handles the state-tracking for connections. Since state tracking is not related to any specific protocol, in this case we need to use -m state before using any of the state module options (--state STATE1[,STATE2,..]).
Please note that the state module exists in two forms, both as a shared object that will be loaded to extend the functionality of iptables, as well as kernel modules that actually handle the stateful tracking and actions (ip_conntrack and ipt_state). You need to use the -m module -form when you want to use some of the matches/actions that come with an extension module. Otherwise iptables won't know what the match/action extension means. The first time when iptables loads an extension module, it will cause the kernel netfilter to load the respective kernel module(s). This loading is done automatically when iptables communicates the new rules to the kernel, not packet by packet basis when netfilter is processing traffic.
- --state ESTABLISHED,RELATED : matches packets which are related to connections which we have initiated to the outside world.
- --state NEW : matches only packets that would create a new "state" entry. Connections coming from the outside world would do this. As we'll see later, there are clearer ways of doing this. Again the -m tcp is not necessary.
- -j REJECT --reject-with icmp-host-prohibited : REJECT was the target that drops packets but generates error messages to the source IP of the dropped packet. Here we also specify what kind of error to generate.
If you're interested what other match specifications and options are available for each module, please consult the manual page of iptables first. The match attributes are grouped according to modules, so it's not that bad. It is however a long manual page, so use the search facility in your pager.
Now, let's return to the task at hand. We're implementing security for a web-server. For one, you might have noticed that the ruleset generated by lokkit didn't contain port 443. You could have enabled it in lokkit, but as we promised before, we will never use lokkit again. What about the other services?
Unnecessary holes in the firewall made by =pn lokkit=:
- IPSec: We will punch the holes ourselves if we need them thank you.
- mDNS: We do not plan to run any software that needs it.
- IPP/UDP: We didn't install printing support. That should have been a good signal for lokkit not to punch holes for it.
By utilizing the mDNS-hole an normal user on our system could run their own software that would listen on port 5353 for incoming traffic and through this remote controlling the system would be possible. Not a good idea. As for the IPSec-hole, listening for arbitrary packets at IP level will require root privileges so it's not so bad. Binding to the IPP port will also require root privileges (port < 1024), so it might not be critical.
However, at this point one needs to be reminded that netfilter will check the chain of rules until it finds a match. This will be done for each and every packet received (for INPUT) and sent (for OUTPUT). In order to make packet delivery faster, it makes sense to rearrange the rules into an order which is likely to minimize amount of matching necessary for "normal" traffic. We'll use this rule when we do our own ruleset.
We want to achieve the following:
- Allow all traffic going outside. In some environments we might want to limit communication outside as well, but in our case we'll keep it simple.
- Allow returning traffic back in. By returning we mean packets that are coming in because we initiated communication originally with the remote end. This is where the stateful firewalling support is done.
- Drop traffic coming from certain networks (as an example).
- Allow clients to connect to our web server (TCP ports 80 and 443).
- Allow us to connect using SSH (TCP port 22)
- Drop all other traffic by default.
- Log packet headers for dropped traffic so that we can tweak our firewall if necessary.
Since we're not going to do forwarding, we'll set the default policy in the FORWARD-chain to DROP. After that we'll have a situation similar to the picture below:
[ Traffic flow with simple netfilter rules ]
Click here for full resolution version (new window)
The picture has two network interface cards just to illustrate that all packets received from the outside world are treated equally before they arrive into netfilter. Also packets coming from the loopback interface use the same packet transfer paths. We should also notice a similar arrangement for packets going out of our system.
By the way, the picture is technically slightly incorrect, but "you don't need to know". Since we're not doing routing, you can assume that this is the location of netfilter. Reality is much uglier but doesn't affect our rules in any way.
We'll next create a new file to hold our own rules. You can start by copying the old one as a template so that you don't have to set the permissions manually afterwards. Normal users must not be able to read the firewall rules file.
We'll start by going over our new ruleset. Notice that as an example, we also include couple of DROP targets for traffic sources that are causing problems for us. This should be done only after you actually experience continued attacks from such networks. There are special modules and programs that can create these kind of rules dynamically for you, but this document is not about advanced netfilter.
# New version of netfilter rules (for iptables)
# Changes wrt the original lokkit-ruleset:
# - Implement rules directly in chains (No RH-Firewall-1)
# - Disable IPSEC
# - Optimize rule order
# - Remove mdns and IPP discovery
# - Change the default rule to DROP for INPUT
# - Deny all routed packets (policy of FORWARD)
# - Last rule in INPUT is logging (for debugging)
*filter
:INPUT DROP [0:0]
:FORWARD DROP [0:0]
:OUTPUT ACCEPT [0:0]
# allow returning packets to come back to us
# (this rule will be used most of the time, so we
# place it as the first one)
-A INPUT -m state --state ESTABLISHED,RELATED -j ACCEPT
# an example how to drop traffic from specific network
# in this case we drop all traffic coming from network
# 10.0.0.0/255.0.0.0 (iptables supports CIDR so we use
# that)
-A INPUT -s 10/8 -j DROP
# an example how to drop traffic but generate error
# messages to sender when dropping happens. this is
# the recommended mechanism when dropping traffic
# that originates from a "trusted" network.
# (i.e., use REJECT target instead of DROP)
-A INPUT -s 172.16/16 -j REJECT
# we place the connection creation for 80 and 443 next
# since this is a webserver and we want to handle these
# quickly.
# note that we don't need to explicitly to use the
# state module (as it was done in the RH default rules).
# also listing -m tcp is unnecessary (as lokkit does).
# in general, -m 'module' is only necessary when you're
# using a match-specifier that a non-built-in module
# provides. Since we don't need anything form 'state'
# and the 'tcp' module is always resident in iptables,
# we leave the -m's out.
-A INPUT -p tcp --dport 80 -j ACCEPT
-A INPUT -p tcp --dport 443 -j ACCEPT
# allow local traffic to reach us
-A INPUT -i lo -j ACCEPT
# allow ECHO requests (ping)
-A INPUT -p icmp --icmp-type ping -j ACCEPT
# allow incoming ssh for administration
-A INPUT -p tcp --dport 22 -j ACCEPT
# log all other traffic so that we can fix this ruleset
# if something doesn't work. You might also want to
# use the 'limit' module (man iptables) to limit the
# amount of logging generated. This is very much
# recommended if you leave logging enabled. Otherwise
# your system is susceptable to denial of service by
# someone filling your disk with logfiles.
-A INPUT -j LOG
# there is no need to DROP anything at the end since
# the policy on INPUT-chain is already DROP
COMMIT
[ Our new ruleset ]
The ruleset has also been organized so that number of rules to match will be minimal for normal traffic. Also note the default policies for the INPUT and FORWARD chains. If you're wondering why we're not using the state module in the new connection rules, there is a simple explanation. Once any packet is accepted (by either OUTPUT or INPUT chains), it will automatically create a new state database entry if it seems a valid connection packet (SYNs for TCP, any for UDP). The state database also has it's own features, one of the important one being timeouts, but these are best explained in the manual page of iptables.
Note that this sequence might break your SSH connection to the system when you implement it. This might happen when you have typos or mistakes in the new configuration. Be prepared to gain physical access (or use a remote control mechanism) to your system.
[user@centos sysconfig]$ sudo cp iptables iptables-new
[user@centos sysconfig]$ sudo vi iptables-new
[user@centos sysconfig]$ sudo su -
Password: Enter the password for your normal account
If you need root access for longer sequences of commands, this
is the correct way of getting a root shell. Remember to exit it
as soon as possible
[root@centos ~]# iptables-restore < /etc/sysconfig/iptables-new
[root@centos ~]# iptables -vnL
Chain INPUT (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
85 6852 ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
0 0 DROP all -- * * 10.0.0.0/8 0.0.0.0/0
0 0 REJECT all -- * * 172.0.0.0/16 0.0.0.0/0 reject-with icmp-port-unreachable
0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:443
0 0 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
0 0 ACCEPT icmp -- * * 0.0.0.0/0 0.0.0.0/0 icmp type 8
0 0 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:22
0 0 LOG all -- * * 0.0.0.0/0 0.0.0.0/0 LOG flags 0 level 4
Chain FORWARD (policy DROP 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 69 packets, 6100 bytes)
pkts bytes target prot opt in out source destination
[root@centos ~]# telnet 192.168.1.30 1234
Trying 192.168.1.30...
telnet: connect to address 192.168.1.30: Connection refused
telnet: Unable to connect to remote host: Connection refused
[root@centos ~]# iptables -vnL
(output cut to include the relevant bits)
3 180 ACCEPT all -- lo * 0.0.0.0/0 0.0.0.0/0
(run "telnet centos 1234" on another host to generate traffic
that our firewall should block)
[root@centos ~]# tail -3 /var/log/messages
Oct 17 19:08:51 centos kernel: IN=eth0 OUT= MAC=00:0c:29:98:a0:a0:00:0e:0c:bc:f7:23:08:00
SRC=192.168.1.3 DST=192.168.1.30 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=33982 DF PROTO=TCP
SPT=35160 DPT=1234 WINDOW=5840 RES=0x00 SYN URGP=0
Oct 17 19:08:55 centos kernel: IN=eth0 OUT= MAC=00:0c:29:98:a0:a0:00:0e:0c:bc:f7:23:08:00
SRC=192.168.1.3 DST=192.168.1.30 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=33983 DF PROTO=TCP
SPT=35160 DPT=1234 WINDOW=5840 RES=0x00 SYN URGP=0
Oct 17 19:09:02 centos kernel: IN=eth0 OUT= MAC=00:0c:29:98:a0:a0:00:0e:0c:bc:f7:23:08:00
SRC=192.168.1.3 DST=192.168.1.30 LEN=60 TOS=0x10 PREC=0x00 TTL=64 ID=33984 DF PROTO=TCP
SPT=35160 DPT=1234 WINDOW=5840 RES=0x00 SYN URGP=0
(proceed by testing telnet centos 80 from another computer so
that we test the tcp/dport-rule instead of the lo-rule. the command
has been left out since by now you should know which tool to use)
GET / HTTP/1.0<enter><enter>
[root@centos ~]# iptables -vnL
(output cut)
Chain INPUT (policy DROP 3 packets, 180 bytes)
pkts bytes target prot opt in out source destination
2100 181K ACCEPT all -- * * 0.0.0.0/0 0.0.0.0/0 state RELATED,ESTABLISHED
0 0 DROP all -- * * 0.0.0.0/8 0.0.0.0/0
0 0 REJECT all -- * * 172.0.0.0/16 0.0.0.0/0 reject-with icmp-port-unreachable
1 60 ACCEPT tcp -- * * 0.0.0.0/0 0.0.0.0/0 tcp dpt:80
(notable here is that only the SYN is matched in the dport-rule, rest go into related)
(you can also repeat the test using https, even with telnet)
[root@centos ~]# exit
logout
[user@centos sysconfig]$ sudo cp iptables-new iptables
[user@centos sysconfig]$ ls -l iptables iptables-new
-rw------- 1 root root 2078 Oct 17 19:14 iptables
-rw------- 1 root root 2078 Oct 17 19:04 iptables-new
[user@centos sysconfig]$ sudo /etc/init.d/iptables stop
Flushing firewall rules: [ OK ]
Setting chains to policy ACCEPT: filter [ OK ]
Unloading iptables modules: [ OK ]
[user@centos sysconfig]$ sudo /sbin/iptables -vnL
Chain INPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain FORWARD (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
Chain OUTPUT (policy ACCEPT 0 packets, 0 bytes)
pkts bytes target prot opt in out source destination
[user@centos sysconfig]$ sudo /etc/init.d/iptables start
Flushing firewall rules: [ OK ]
Setting chains to policy ACCEPT: filter [ OK ]
Unloading iptables modules: [ OK ]
Applying iptables firewall rules: [ OK ]
[user@centos sysconfig]$ sudo /sbin/iptables -vnL
(output cut, you should see your new firewall ruleset)
[ Implementing and testing the new firewall ruleset ]
New commands:
- sudo su - : Since we can run any commands as root using sudo, we utilize that to start a new root shell. There are also other ways of doing this (via sudo directly).
- iptables-restore < filename : Uses iptables to flush all rules and load the new rules from standard input. Shell redirection was the reason why we switched to root shell before.
- /etc/init.d/iptables stop : Since iptables is not really a service, the service script has another purpose. It will be run with start on system boot and with stop on system shutdown. Internally it uses iptables-restore and iptables -F to load the ruleset and to flush the rules (respectively). This is a quick way to disable the kernel firewall completely, but remember to enable it after you're done.
That's it!
After you've done this couple of times, it won't seem overly complex or hard. You should always have a working ruleset that you trust handy, so that you can start with that as a template. The one generated by lokkit might not be the best one for this, since it contains the unnecessary holes.
If you're not connected to a real IPv6 network (most of us are not), you might want to disable the protocol completely. There is a separate tool (normally called iptables6) to modify the netfilter rules for IPv6 but it might not even be installed. This will mean that your system will accept all traffic coming in when the protocol is non-IPv4, including IPv6 traffic. Since IPv6 traffic is routable, this might expose your system and services to unnecessary risks.
Normally your router would filter out non-IPv4 traffic, but it might make sense to remove the whole protocol from your server so that you don't leave an easily exploitable back door by mistake.
We'll start by checking whether our interfaces have the automatic IPv6 link-local address. This can be seen by ifconfig or the more modern ip tool. In most Linux-distributions the IPv6 support is built in as a module, so that normally one could remove the module during runtime to disable the protocol. We'll try removing the module first. The module that implements the IPv6 protocol is called ipv6.
[user@centos ~]$ /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:98:A0:A0
inet addr:192.168.1.30 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe98:a0a0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:13060 errors:0 dropped:0 overruns:0 frame:0
TX packets:996 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:820150 (800.9 KiB) TX bytes:148514 (145.0 KiB)
Interrupt:177 Base address:0x1400
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
inet6 addr: ::1/128 Scope:Host
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:304 (304.0 b) TX bytes:304 (304.0 b)
[user@centos ~]$ /sbin/lsmod | grep ipv6
ipv6 241761 16
[user@centos ~]$ sudo /sbin/modprobe -r ipv6
FATAL: Module ipv6 is in use.
[ Checking for IPv6 and trying removal ]
Removing the module using modprobe doesn't seem to work. In some (rare) cases modules can only be loaded into the kernel, but not removed. This is the case with some hardware drivers as well. Seems that we've stumbled upon such a driver.
There are several techniques to disable a specific module:
- Rebuilding the kernel not to contain the driver at all (either built in in the kernel or as module). Rebuilding kernels is not covered here and because of that, this is not the recommended way.
- Rename the file which contains the driver. The problem with this approach is that once you upgrade your kernel package, it will contain an updated version of the "disabled" module as well. So you'd need to remember to rename the file after each kernel upgrade. This is less than elegant.
- Remove the mechanism that causes the module to load. In some cases it is relatively easy to locate the exact system startup script that will load the driver in question (look for files that use modprobe). For ipv6, this won't work since it is not activated by any script.
- Tell modprobe to disable "load on feature-request"-mechanism. This is the mechanism that we'll use, although it is not perfect either (as we'll see).
Disabling the ipv6 driver is problematic because it's not possible to locate the mechanism that causes the driver to load. Obviously it will load at some point since when we start our system, ifconfig will show the IPv6 addresses for our interfaces. What we have encountered in fact is the kernel automatic driver loading mechanism. Historically this was called kmod (and some people still use this name). The feature is quite nifty and useful (normally).
When some software component on our system (user-space application, another software driver, some hardware driver) requires a feature from the kernel, the kernel will first check whether the feature is enabled. If not, it will ask modprobe to load some driver that will implement this feature. Once modprobe returns, the kernel will recheck the feature list and mark the feature either present or disabled. You might be asking "hold on, the kernel will start modprobe? But modprobe is a normal application that is run from the shell?". Indeed. Normally applications are started from user-space by users (or administrators). In Linux it is also possible for the kernel to start a program (why not). This feature is used also in couple of other places (the "hotplug"-mechanism that tells udevd that hw-configuration has changed, and so on). Most people just assume that these things happen by magic and this causes some confusion.
The reason why ipv6 is loaded into our system is that something requires the protocol (for one reason or another). It might well be that some software has been built with IPv6-support (most networking software is) and when that software starts, it will open a TCP-socket. This in turn will cause the TCP-driver inside the kernel to request for both IPv4 and IPv6 protocols and since IPv6 feature is missing, the kernel will ask modprobe to do something about it.
Depending on the version of modprobe that you have, the configuration file(s) that it reads each time someone runs it (including the kernel) will be different. For older versions (that RHEL4 and Centos use), there is only one file: /etc/modprobe.conf . This is the file where we'll disable the IPv6 protocol feature. Since unloading the ipv6 module is impossible, the only way to test this properly is to reboot the whole system. We'll do a reboot and the continue with couple of other tests.
The new contents of /etc/modprobe.conf in our test system is like this:
alias eth0 pcnet32
alias scsi_hostadapter mptbase
alias scsi_hostadapter1 mptscsi
alias scsi_hostadapter2 mptfc
alias scsi_hostadapter3 mptspi
alias scsi_hostadapter4 mptsas
alias scsi_hostadapter5 mptscsih
# disable ipv6 (do not touch the existing lines in the file!)
# the alias line stops modprobe from automatically loading the
# ipv6 module (protocol filter 10).
# the alias line doesn't stop manual loading
alias net-pf-10 off
[ Modified configuration file ]
Note that you should only add one line to disable the IPv6 driver. Do not touch the other lines that you might have in the file! The number given in Linux for the IPv6 protocol "filter" is 10 and we use the alias command to map this feature into a driver called off (which doesn't really exist, but is a magical keyword for modprobe).
[user@centos ~]$ uname -r
2.6.9-42.ELsmp
[user@centos ~]$ find /lib/modules/2.6.9-42.ELsmp/ -type f -name "ipv6*"
/lib/modules/2.6.9-42.ELsmp/kernel/net/ipv6/ipv6.ko
[user@centos ~]$ sudo vi /etc/modprobe.conf
[user@centos ~]$ sudo reboot
(reboot happens)
[user@centos ~]$ /sbin/ifconfig
eth0 Link encap:Ethernet HWaddr 00:0C:29:98:A0:A0
inet addr:192.168.1.30 Bcast:192.168.1.255 Mask:255.255.255.0
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:87 errors:0 dropped:0 overruns:0 frame:0
TX packets:67 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:10229 (9.9 KiB) TX bytes:8077 (7.8 KiB)
Interrupt:177 Base address:0x1400
lo Link encap:Local Loopback
inet addr:127.0.0.1 Mask:255.0.0.0
UP LOOPBACK RUNNING MTU:16436 Metric:1
RX packets:4 errors:0 dropped:0 overruns:0 frame:0
TX packets:4 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:0
RX bytes:304 (304.0 b) TX bytes:304 (304.0 b)
[user@centos ~]$ sudo /sbin/modprobe ipv6
Password:
[user@centos ~]$ /sbin/ifconfig eth0
eth0 Link encap:Ethernet HWaddr 00:0C:29:98:A0:A0
inet addr:192.168.1.30 Bcast:192.168.1.255 Mask:255.255.255.0
inet6 addr: fe80::20c:29ff:fe98:a0a0/64 Scope:Link
UP BROADCAST RUNNING MULTICAST MTU:1500 Metric:1
RX packets:212 errors:0 dropped:0 overruns:0 frame:0
TX packets:145 errors:0 dropped:0 overruns:0 carrier:0
collisions:0 txqueuelen:1000
RX bytes:21017 (20.5 KiB) TX bytes:17841 (17.4 KiB)
Interrupt:177 Base address:0x1400
[user@centos ~]$ sudo /sbin/modprobe -r ipv6
FATAL: Module ipv6 is in use.
[ Modifying modprobe configuration and checking ]
We started off by showing how to locate the module file if you think that renaming the driver would suit you better. We won't rename in this case. We then implement the feature-request-modification, reboot and check that the interfaces do not have IPv6 addresses. As you can see, using modprobe directly with the module name still loads the driver. And again, we cannot unload the driver after loading it (IPv6 is "special") so we'd need to reboot the system after this.
The only way to stop the loading of the driver by name is renaming or deleting the driver (which obviously is not recommended since it will break the kernel package information in the package database).
You might be interested in seeing what other protocols are loaded and from which modules. Start with the /proc/net/protocols file to see which protocols are defined in the kernel. Unfortunately this file doesn't list the pf-number, so you'll have to seek wisdom in another file: /usr/include/bits/socket.h (or google). For example, if you search the header file for PF_INET6, you will see that it is defined as 10. This number is not the same as the Ethernet protocol field. Instead the number is an unique identifier for something called "socket family" identifying what "kind" of socket a program may use for exchanging data. You cannot disable modules that are marked as kernel in the proc-file (since they're built in into the kernel, not as modules). By the way, if your socket.h is missing, it means that you haven't installed the development package for libc. This package is necessary to build C language software on your system.
SELinux is a rather complicated mechanism which can be used to specify whether operations done by user space processes are allowed or not, depending on the context. Since a lot of system resources can be accessed via device files (and other files), SELinux also provides mechanisms at that level. Besides files, SELinux allows you to specify all kinds of system call restrictions but modifying those is quite complex as it will require a lot of UNIX-specific knowledge on how network (and other) services are created.
The SELinux configuration is located in many places in the system but the most important ones are the extended attributes related to files. When a program is started, the kernel will "load" the program and the SELinux subsystem will try to load its extended attributes for the program file. These attributes will contain SELinux-specific control data that will tell what kind of security restrictions will be placed on the process that is created.
The rules are grouped into different execution domains so that one doesn't need to specify each rule multiple times for a group of related processes. These groups are called contexts. Context configuration is also stored somewhere on the disk and will be loaded into the kernel. The context configuration basically specifies what kind of operations are allowed for processes executing in that context and what kind of operations may cross into other contexts. Most of the time one will be modifying the context-information on existing files and directories so that some network service may access the file. This mechanism is in addition to the regular UNIX classical access rights as well as the "POSIX" ACLs.
Creating new contexts and specifications is outside the scope of this document, but http://www.redhat.com/docs contains more information on this. SELinux is a moving target so reader discretion is advised.
Instead, Red Hat provides you with predefined templates which consist of easily manipulated "flags" that can be on or off. After modifying the template, it will be applied to the relevant files and locations in the system so that the changes will take an effect after the service in question will start again. Some of the settings may take effect instantaneously, but the recommended way of doing the template changes is stopping the service, doing the modifications and then starting it again. Network services packaged by Red Hat come with relevant SELinux templates so editing templates is normally what is done with Red Hat. Other distributions do not use SELinux by default (with the exception of Centos and Fedora Core for obvious reasons). Debian and Ubuntu have SELinux support in them, but no service templates (i.e., SELinux is not integrated properly into them).
It is recommended that you use the graphical program system-config-securitylevel to modify SELinux settings. It is also possible to edit the templates using a terminal connection, but that is not covered here.
You might recall that we used the same tool to "setup" the "firewall". Do not touch the firewalling settings since you already have a working firewall config. Instead select the tab marked "SELinux".
[ Starting the securitylevel-tool ]
[ General SELinux settings ]
The SELinux-tab contains the general SELinux-settings most important of which are the Enabled and Enforcing check-boxes. When both are set, all SELinux rules are enforced by the kernel which means that the rules cannot be avoided by the services/processes targeted by the rules. Removing the Enforcing check will allow processes to bypass the rules, but each violation will be logged (grep for AVC in your security logs). This setting is sometimes suitable when you're testing some new service or extension. You can disable all of SELinux by un-checking the Enabled check-box obviously. Sometimes it it useful to boot the system without SELinux enabled for temporary administrative work. In this case you may pass selinux=0 option to the kernel (in boot manager).
The installed SELinux templates are displayed in the tree-view from where you can see which network services can be easily modified using this tool. As a short example, we'll show the settings relating to our web server.
[ Apache HTTP-server related template flags ]
You might want to remove some of the flags, but it is better first to modify Apache HTTP-server configuration and only after the new restricted configuration works, modify the SELinux template flags.
[ Options and commands related to context manipulation ]
Finally we have a screenshot of SELinux extended attributes and how to manipulate them. Both ps and ls have a special option -Z which will show the process context or file context and you can use chcon to copy context information between files. There are some 10 other SELinux-related command line tools as well but these are better described in the Red Hat documentation PDFs.
In conclusion, always enable SELinux when deploying network services. SELinux will protect the rest of your system when some network service is compromised (even if it is running as root). Besides just relying on SELinux you also need to keep your system up to date. SELinux is not some magical panacea to all security related problems, it just adds some protection to well designed and administered systems.
I'm not a native English speaker (as you might have noticed based on text) and for this reason I'd like to thank the following person(s) for having the courage and the will-power to read this blurb and provide me with feedback.
Thank yous (in no particular order):
- Mikko Ruuska
- Atro Tossavainen
Document history
- Pre 2006-11-09: Main dev, pictures etc.
- 2006-11-09: Publish (pre-version).
- 2006-11-12: Notes on where the 10 in IPv6 comes from (thanks Mikko!). Typos/clarifications.