FreeBSD jails with ezjail

From Secure Computing Wiki
Revision as of 07:38, 27 September 2007 by Ecrist (Talk | contribs) (Building Our ezjail Flavour)

Jump to: navigation, search

At work, we've decided to virtualize all of our FreeBSD production systems with jails. We're hoping this will help with upgrades and potential system failure, as we'll be to move virtual systems around, where neeeded, with relative ease.

While setting up jails with FreeBSD directly isn't a difficult task, there is one major limitation, if you're going to have multiple jails on the same host. The entire base system (/bin, /usr/bin, etc), needs to reside within each jail. Using a very well written set of shell scripts to help manage our jails, however, we're able to mount, with the help of nullfs in FreeBSD 6+, read-only, our entire base system.

Here, I'll describe in great detail the entire process I used to install ezjail, build our basejail, and create a flavour [sic] that meets our needs.

Build host system

Install/Update FreeBSD

To begin, we build out host FreeBSD system. This includes performing a full installation of FreeBSD 6.2, and doing all the updates. Whereas freebsd-update is a rather nice binary update tool, it is advisable to build/install directly from source, and you'll end up doing it for your jails anyways. You won't be saving yourself any time doing binary updates.

Install Common Ports

Once we have a basic install of FreeBSD, our network requires we install postfix with sasl to send email. This allows email to be sent to a user with a simpel 'mail <user>' and everything will get translated correctly. Along with setting up mail, continue with installing any ports that you'll need/want on all your jails, onto the base system. Individual jail ports, such as apache, mysql, etc, should not be installed at this time. Only ports you want on all jails should be installed.

We've installed the following ports:

  • sudo (security/sudo)
  • vim-lite (editors/vim-lite)
  • bash (shells/bash)
  • rsync (net/rsync)

Install ezjail

Now install the ezjail scripts from the FreeBSD ports tree at sysutils/ezjail. You will be left with an script, an script, and an rc script in /usr/local/etc/rc.d. To enable our jails to start at system boot, we needed to add ezjail_enable="YES" to /etc/rc.conf:

echo 'ezjail_enable="YES"' >> /etc/rc.conf


There is a configuration file for ezjail in /usr/local/etc/ezjail.conf.sample, we want to move that to /usr/local/etc/ezjail.conf and edit the file. Make appropriate changes for your network. We run our own CVS mirror, so our file looks like this:


# Location of the tiny skeleton jail template

# Location of the huge base jail

# Location of your copy of FreeBSD's source tree

# In case you want to provide a copy of ports tree in base jail, set this to
# a cvsroot near you

# This is where the install sub command defaults to fetch its packages from

# base jail will provide a soft link from /usr/bin/perl to /usr/local/bin/perl
# to accomodate all scripts using '#!/usr/bin/perl'...

Change <our_local_FTP/CVS_mirror> for your correct values.

Build basejail

At this time, we should be ready to build our basejail!

Run the following command to build basejail within your configured jail home:

ezjail-admin update -i
  • The -i option above tells ezjail that we've already built-world (when we updated FreeBSD on the host system), so it simply does a make installworld to your jail home. Omitting the -i causes this process to take a considerable amount of time.

When this process is complete, you should have a directory structure similar to this in your jail home (/usr/jails by default):

drwxr-xr-x   5 root  wheel   512B Sep 26 14:57 .
drwxr-xr-x  18 root  wheel   512B Sep 25 13:11 ..
drwxr-xr-x   9 root  wheel   512B Sep 26 13:42 basejail
drwxr-xr-x   4 root  wheel   512B Sep 26 14:43 flavours
drwxr-xr-x  12 root  wheel   512B Sep 26 13:58 newjail

If yours checks out, we're ready to start building our localized flavour!

Building Our ezjail Flavour

Version 1.2 of ezjail introduces the concept of Flavours. A flavour corresponds to a preconfigured Jail. Basically it is a directory being copied recursively over a new Jails root and a script being run on its first startup to do some initialization.

Copy /usr/local

Assuming your jail home is /usr/jails (which we will assume for the remainder), you should have a /usr/jails/flavours/default directory. We want to copy that directory to a name for our own customized flavour. Ours will be called clx.

 cp -Rp ./default ./clx

When your jail is finally built, anything within the corresponding flavour directory will be copied over the basejail that is installed. Our first order of business is to copy our entire host's /usr/local directory into our flavour directory. This will get us all those common ports we installed earlier:

cp -Rp /usr/local/* ./clx/usr/local/
  • Note, this process will probably take quite a while, depending on which ports you have installed.

Create /etc/<config> files

Your jails, by default, have a very limited, and very incorrect setup. Here are the specific files we had to copy from our host /etc/ to our flavour's etc:

File Reason
/etc/localtime Puts our jail in the correct timezone.
/etc/resolv.conf Allows our system to resolve domain names and URLs.
/etc/motd Create's a login banner for ssh users.
/etc/shells Since we installed bash, we need an appropriate shells file.

In addition to the copied files, we made some extensive changes to /etc/periodic.conf and /etc/rc.conf. First, rc.conf:

# Miscellaneous Configuration
network_interfaces="lo0"                # No network interfaces aside from the loopback device
kern_securelevel_enable="YES"           # Enable 'securelevel' kernel security
kern_securelevel="1"                    # See init(8)
rpcbind_enable="NO"                     # Disable RPC daemon
cron_flags="$cron_flags -J 15"          # Prevent lots of jails running cron jobs at the same time
syslogd_flags="-ss"                     # Disable syslogd listening for incoming connections
sendmail_enable="NONE"                  # Comppletely disable sendmail
clear_tmp_enable="YES"                  # Clear /tmp at startup

## Mail Config
postfix_enable="YES"                    # Enable postfix at boot.
sendmail_enable="NO"                    # Disable Sendmail
sendmail_submit_enable="NO"             # Disable sendmail submit
sendmail_outbound_enable="NO"           # Disable sendmail outbound
sendmail_msp_queue_enable="NO"          # Disable sendmail msp queing

# SSHD Configuration
sshd_enable="YES"                       # Enable sshd
  • Note that our rc.conf doesn't contain any IP address, etc. This is to be handled entirely by the host system.

Next, our periodic.conf file:

weekly_whatis_enable="NO"       # our jails are read-only /usr




#monthly_show_info="NO" # Show login accounting