experimentalworks

I usually want to have my disks encrypted. As all my storage servers run on recent FreeBSDs I usually use GEOM ELI, also known as GELI, to encrypt the disks. Geli is pretty straight forward, runs out of the box on FreeBSD and is easy to set up. So when updating my storage by buying two 500GB disks, I wanted to checkout the ZFS implementation on FreeBSD which is available, but still experimental, on FreeBSD 7.0.

What is ZFS?
ZFS is more than just a filesystem. While usually volume manager and filesystems are separated, Sun’s ZFS combines both. It gives you one command to do both, setting up RAID configuration, setting mountpoints and creating filesystems. In addition to that it can address uo 16 Exbibytes. What makes ZFS sexy is that it gives you a very powerful but easy way to set up RAID-0, RAID-1 or RAID-5 (in fact RAID-Z) with just one command. You can also export and import filesystems and creating snapshots, and just move the complete filesystem to another location. Just take a look at the various ZFS related sites about the featureset. It is planned to implement disk encryption on ZFS, but at the moment neither the Sun Solaris implementation nor the FreeBSD implementation support encryption of ZFS filesystems.

Luckily FreeBSD includes a rock solid, easy to use block device encryption that can be used to encrypt disks on a lower level than the filesystem. We are going to use this interface, called GELI to setup our encrypted ZFS.

So first of all, we prepare our keys that will be used to decrypt and encrypt our disks. You can separate keys into a keyfile and a password. Only the combination of both can decrypt the disk.

#dd if=/dev/random of=/root/ad4.key bs=64 count=1
#dd if=/dev/random of=/root/ad6.key bs=64 count=1

We now initialize our encryption with our generated keys. You will be promoted for a password

# geli init -s 4096 -K /root/ad4.key /dev/ad4
Password:
# geli init -s 4096 -K /root/ad6.key /dev/ad6
Password:

And attach them, so that we can write on the disk. With attaching, we start decrypting the content of the disks so it is accessable to the filesystem.

# geli attach -k /root/ad4.key /dev/ad4
# geli attach -k /root/ad6.key /dev/ad6

We know have two new devices: /dev/ad4.eli and /dev/ad6.eli. /dev/ad4 and /dev/ad6 is encrypted, while /dev/ad4.eli and /dev/ad6.eli is decryped.
We can work with the later one like with every other disk.

If you want to make things a little bit safer, so that you are sure no old content is on the disk, rewrite the complete disk
using dd.

# dd if=/dev/random of=/dev/ad4.eli bs=1m
# dd if=/dev/random of=/dev/ad6.eli bs=1m

Now create our ZFS mirror on top of that

# zpool create -f tank mirror ad4.eli ad6.eli

We use the -f switch here if your disks differ in size a little bit (this will usually happen if you work usb sticks or something equal)

After that, we have to setup the necessary startup files, so that we will be prompted for the password on the next boot and that the ZFS will be mounted automaticly

In /boot/loader.conf

zfs_load=”YES”
vm.kmem_size_max=”1073741824″
vm.kmem_size=”1073741824″

The later two options are necessary as ZFS eats a lot of kernel memory (see ZFS Tuning Guide: for more details)

In /etc/rc.conf

geli_devices=”ad4 ad6″
geli_ad4_flags=”-k /root/ad4.key”
geli_ad6_flags=”-k /root/ad6.key”
geli_autodetach=”NO”
zfs_enable=”YES”

The zfs_enable switch will cause ZFS to mount the partitions on startup. The geli_devices=”ad4 ad6″ will cause geli to setup both devices on startup while the appropriate flags help to find the keyfile for the decryption.

Notice that geli_autodetach MUST be NO. Using ZFS geli sometimes detaches even if ZFS writes on the devices, causing an anoying core dump.

After that, just reboot, you will be asked for the password of your disk. If your finished booting, your ZFS will be available like every other file system.
Type

# zpool status
pool: tank
state: ONLINE
scrub: none requested
config:

NAME STATE READ WRITE CKSUM
tank ONLINE 0 0 0
mirror ONLINE 0 0 0
ad4.eli ONLINE 0 0 0
ad6.eli ONLINE 0 0 0

errors: No known data errors

to see the status of your ZFS system.