Home→Archive↴

Install NetBSD 10 with (almost) full disk encryption

Last edited on 2024-09-08 • Tagged under  #netbsd   #bsd   #encrypt 

The first time I installed NetBSD I used sysinst, a menu-based program launched at boot that runs in the console. It has a clear and concise layout and I was quickly up-and-running on my new BSD system.

For my second time installing NetBSD I wanted to include full disk encryption to protect personal data in case the device is lost or stolen. Its not really enough to simply encrypt home directories. Passphrases and sensitive data can linger and be extracted from locations such as system logs and swap memory. There is a trade-off to be made between how much to encrypt, the convenience of operating the system, and the ability for the system to boot.

NetBSD ascii logo

The arrangement outlined below uses an unencrypted EFI system partition (UEFI) plus a minimal root filesystem that boots up to an encryption passphrase prompt, which upon successful entry proceeds to unlock an encrypted partition containing var, swap, usr, and home.

Sysinst does not provide the option for encrypting the system in this manner, so early in the install process I switch to /bin/sh and proceed to manually install NetBSD.

This is how I do it...

1. Let's go!

1.1 Setup

  • Target device is x86_64 architecture using UEFI boot
  • NetBSD is the sole OS on a single 1TB NVMe disk
  • GPT partition table with 3 partitions (wedges):
    • dk0 = Size: 128MB; Format: msdos; Use as: EFI system partition
    • dk1 = Size: 6GB; Format: ffs; Use as: root partition
    • dk2 = Size: ->END; Format: cgd; Use as: encrypted partition
  • Within the encrypted partition 4 disklabels are created:
    • cgd0a for /var; Size: 6GB
    • cgd0b for swap; Size: 4GB
    • cgd0e for /usr; Size: 40GB
    • cgd0f for /home; Size: ->END

1.2 Download

The latest install images are available here: pub/NetBSD

Download the NetBSD-10.0-amd64-install.img.gz install image and SHA512 hash file.

Verify image integrity:

$ sha512sum -c --ignore-missing SHA512
NetBSD-10.0-amd64-install.img.gz: OK

1.3 Prepare install media

Note: Installer was prepared on a Linux system.

Unpack install image:

$ gunzip NetBSD-10.0-amd64-install.img.gz

Write installer to an unmounted USB storage device using the dd command as root.

BE VERY CAREFUL TO NOTE THE PROPER DEVICE. ALL DATA ON THE DEVICE WILL BE OVERWRITTEN.

Example: On a Linux system, if a USB stick appears as sdx1, then write the installer to sdx (omit the partition number):

# dd if=/path/to/NetBSD-10.0-amd64-install.img of=/dev/sdx bs=1M
$ sync

2. Configure live environment

2.1 Boot

After the installer has successfully booted into the NetBSD kernel, the menu-based sysinst program is launched.

Select the desired language and keyboard type.

2.2 Network

Select Utility menu then Configure network.

Example: On my system I have two interfaces detected: wn0 (wired) and iwn0 (wireless).

I choose to configure the wired ethernet interface with autoconfiguration.

2.3 (Optional) Remote login to installer

I make this manual install process easier (cut-n-paste commands; lookup info online) by remotely logging into the installer via SSH from another computer.

Setup SSH on the installer by navigating back to the Utility menu and selecting Run /bin/sh.

Create password for root:

# passwd root

Modify /etc/ssh/sshd_config to allow root to login with password:

PermitRootLogin yes

Start sshd:

# /etc/rc.d/sshd onestart

Retrieve IP address for active interface (example: iwm0):

# ifconfig iwm0

Login to the target device via SSH:

$ ssh root@<target_device_ip_address>

3. Prepare disk

3.1 Identify disks and partitions

Discover what disk devices and partitions have been recognized by the kernel using sysctl(8):

# sysctl hw.disknames
hw.disknames = ld0 dk0 dk1 dk2 dk3 sd0 dk4 dk5 sd1

NVMe devices show up as ld and hard disks are identified by wd. USB devices usually show up as sd.

The dk devices are wedges (partitions) on the storage devices, and this early after boot are usually displayed in order, that is: dk0 through dk3 are wedges on the NVMe target device ld0, and dk4 and dk5 on the USB installer sd0.

Verify by asking for a list of wedges on sd0 using dkctl(8):

# dkctl sd0 listwedges
/dev/rsd0: 2 wedges:
dk4: EFI system, 262144 blocks at 2048, type: msdos
dk5: 30c4cc4e-5369-449c-8994-a4b1ea665b4b, 4853760 blocks at 264192, type: ffs

Check which device the installer booted from using dmesg(8):

# dmesg | fgrep "root on"
[     4.158650] root on dk5

More details about individual disks can be retrieved by extracting parts of the kernel output from the dmesg output (example: for ld0):

# dmesg | fgrep ld0
[     1.045032] ld0 at nvme0 nsid 1
[     1.045032] ld0: 931 GB, 121601 cyl, 255 head, 63 sec, 512 bytes/sect x 1953525168 sectors
[     2.178653] ld0: GPT GUID: 2271c564-dd66-4398-b6f3-12bec0c9df07
[     2.178653] dk0 at ld0: "esp", 614400 blocks at 2048, type: msdos
[     2.248652] dk1 at ld0: "boot", 2097152 blocks at 616448, type: ext2fs
[     2.308652] dk2 at ld0: "root", 134217728 blocks at 2713600, type: <unknown>
[     2.378653] dk3 at ld0: "home", 1816573327 blocks at 136931328, type: <unknown>

There is also the nvmectl(8) command (use atactl(8) for SATA drives):

# nvmectl identify nvme0 | egrep 'Model|Device type|Capacity'
Model Number:               Samsung SSD 980 1TB

3.2 Set disk variable

For the purposes of this HOWTO, the NVMe storage device detected as ld0 is where NetBSD will be installed.

Set the $disk variable:

# disk="ld0"

3.3 Create GPT partition table

Target device uses UEFI to boot and will use GPT partitions:

# gpt destroy $disk
# gpt create -f $disk

3.4 Create partitions

# gpt add -a 2m -s 128m -t efi -l "efi-system" $disk
# gpt add -a 2m -s 6g -t ffs -l "netbsd-root" $disk
# gpt add -a 2m -t cgd -l "syscgd" $disk

Verify the result:

# gpt show $disk
       start        size  index  contents
           0           1         PMBR
           1           1         Pri GPT header
           2          32         Pri GPT table
          34        4062         Unused
        4096      262144      1  GPT part - EFI System
      266240    12582912      2  GPT part - NetBSD FFSv1/FFSv2
    12849152  1940672512      3  GPT part - NetBSD Cryptographic Disk
  1953521664        3471         Unused
  1953525135          32         Sec GPT table
  1953525167           1         Sec GPT header

List wedges:

# dkctl $disk listwedges
/dev/rld0: 3 wedges:
dk0: efi-system, 262144 blocks at 4096, type: msdos
dk1: netbsd-root, 12582912 blocks at 266240, type: ffs
dk2: syscgd, 1940672512 blocks at 12849152, type: cgd

3.5 Format partitions

Format the EFI partition:

# newfs_msdos /dev/rdk0

Format the root partition:

# newfs -O 2 dk1

3.6 Add EFI boot entries

# mount /dev/dk0 /mnt
# mkdir -p /mnt/EFI/boot
# cp -v /usr/mdec/*.efi /mnt/EFI/boot
/usr/mdec/bootia32.efi -> /mnt/EFI/boot/bootia32.efi
/usr/mdec/bootx64.efi -> /mnt/EFI/boot/bootx64.efi
# umount /mnt

3.7 Mount future root

# mount /dev/dk1 /targetroot

4. Encryption

4.1 Create CGD config

NetBSD uses the cryptographic device driver (CGD) to create and manage encrypted storage.

Using cgdconfig(8), a parameters file is generated that stores the encryption type, key length, and a random password salt for the new cgd.

There are a few different encryption ciphers supported. I choose aes-xts with a 512-bit key:

# mkdir -p /targetroot/etc/cgd
# chmod 700 /targetroot/etc/cgd
# cgdconfig -g -V disklabel -o /targetroot/etc/cgd/syscgd aes-xts 512

4.2 Create CGD drive

Create the encrypted storage and assign it a passphrase. This passphrase will be used to open the CGD drive at boot.

Example: NAME is the label used when the GPT partition for CGD was created earlier (syscgd):

# cgdconfig -V re-enter cgd0 NAME=syscgd /targetroot/etc/cgd/syscgd

4.3 Create disklabels

Note: Disklabels c and d have special meaning in NetBSD (addressing the whole disk) and should not be used.

In my example setup, I use:

  • cgd0a for /var; Size: 6GB
  • cgd0b for swap; Size: 4GB
  • cgd0e for /usr; Size: 40GB
  • cgd0f for /home; Size: ->END

Using disklabel(8) in interactive mode:

# disklabel -Ii cgd0
Enter '?' for help
...

Setup /var:

partition>a
Filesystem type [4.2BSD]: 
Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: 
Partition size ('$' for all remaining) [947594c, 1940672512s, 947594M]: 6G
 a:  12582912         0     4.2BSD      0     0     0  # (Cyl.      0 -   6143)

Setup swap:

partition>b
Filesystem type [unused]: swap
Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: a
Partition size ('$' for all remaining) [0c, 0s, 0M]: 4G    
 b:   8388608  12582912       swap                     # (Cyl.   6144 -  10239)

Setup /usr:

partition>e
Filesystem type [unused]: 4.2BSD
Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: b
Partition size ('$' for all remaining) [0c, 0s, 0M]: 40G
 e:  83886080  20971520     4.2BSD      0     0     0  # (Cyl.  10240 -  51199)

Setup /home:

partition>f
Filesystem type [unused]: 4.2BSD
Start offset ('x' to start after partition 'x') [0c, 0s, 0M]: e
Partition size ('$' for all remaining) [0c, 0s, 0M]: $
 f: 1835814912 104857600     4.2BSD      0     0     0  # (Cyl.  51200 - 947593)

If satisfied with the edits, write the label and exit:

partition>W
Label disk [n]?y
Label written
partition>Q

4.4 Verify CGD drive

Copy the config to the target device:

# echo 'cgd0 NAME=syscgd /etc/cgd/syscgd' > /targetroot/etc/cgd/cgd.conf

Close the CGD drive:

# cgdconfig -u cgd0

Unlock it again with the passphrase set earlier:

# cgdconfig cgd0 NAME=syscgd /targetroot/etc/cgd/syscgd

If everything worked, the cgd0 drive is now open and disklabel is visible with:

# disklabel cgd0
# /dev/rcgd0:
type: cgd
disk: cgd0
label: fictitious
flags:
bytes/sector: 512
sectors/track: 2048
tracks/cylinder: 1
sectors/cylinder: 2048
cylinders: 947594
total sectors: 1940672512
rpm: 3600
interleave: 1
trackskew: 0
cylinderskew: 0
headswitch: 0		# microseconds
track-to-track seek: 0	# microseconds
drivedata: 0 

6 partitions:
#        size    offset     fstype [fsize bsize cpg/sgs]
 a:  12582912         0     4.2BSD      0     0     0  # (Cyl.      0 -   6143)
 b:   8388608  12582912       swap                     # (Cyl.   6144 -  10239)
 d: 1940672512         0     unused      0     0        # (Cyl.      0 - 947593)
 e:  83886080  20971520     4.2BSD      0     0     0  # (Cyl.  10240 -  51199)
 f: 1835814912 104857600     4.2BSD      0     0     0  # (Cyl.  51200 - 947593)

4.5 Format and mount

# newfs -O 2 cgd0a
# newfs -O 2 cgd0e
# newfs -O 2 cgd0f
# mkdir /targetroot/var /targetroot/usr /targetroot/home
# mount /dev/cgd0a /targetroot/var
# mount /dev/cgd0e /targetroot/usr
# mount /dev/cgd0f /targetroot/home

5. Install NetBSD

The new system is composed of sets (collections of packages) installed to the target device. These sets are located in /amd64/binary/sets.

At a minimum, you must select a kernel and the base and etc sets. Below are the sets I choose to install.

Note: The tar flag p is very important. It ensures that all files preserve their owners and mode.

# cd /amd64/binary/sets
# for set in base comp etc games gpufw kern-GENERIC man misc modules tests text xbase xcomp xetc xfont xserver; do
> tar -xvzpf $set.tar.xz -C /targetroot
> done

6. Configure NetBSD

6.1 Chroot

Enter chroot on the target device:

# chroot /targetroot

6.2 Create directories

Create the kern and proc directories:

# mkdir kern proc

6.3 Create devices

# cd dev
# sh MAKEDEV all

Note: On a previous install, after rebooting the boot process halted with the error message:

/etc/defaults/rc.conf: cannot create /dev/null: read-only file system
/etc/rc: cannot create /dev/null: read-only file system

MAKEDEV had created null but it was incorrectly configured:

-rw-r--r--  1 root  wheel               0 Sep  4 16:57 null

Fix: To avoid this error, remove the existing null:

# rm /dev/null

Re-create null with mknod(8):

# mknod -m 0666 -u root -g wheel /dev/null c 2 2

Show:

# ls -l null
crw-rw-rw-  1 root  wheel  2, 2 Sep  27 16:59 null

6.4 Create fstab

# cat > /etc/fstab << EOF
NAME=netbsd-root  /         ffs     rw      1 1
tmpfs             /tmp      tmpfs   rw,-m=1777,-s=ram%25
kernfs            /kern     kernfs  rw
ptyfs             /dev/pts  ptyfs   rw
procfs            /proc     procfs  rw
tmpfs             /var/shm  tmpfs   rw,-m1777,-sram%25
# Encrypted file-systems
/dev/cgd0a        /var      ffs     rw      1 2
/dev/cgd0b        none      swap    sw
/dev/cgd0e        /usr      ffs     rw      1 2
/dev/cgd0f        /home     ffs     rw      1 2 
EOF

6.5 Edit rc.conf

# vi /etc/rc.conf

Note: In this example, my hostname was earlier set to demobsd.localdomain during network setup, and my wired interface is wm0. Adjust accordingly.

Settings:

# Set this to "YES", otherwise system will boot in single-user mode.
rc_configured=YES

# Add local overrides below. Wait for CGD to be unlocked before mounting.
critical_filesystems_local="OPTIONAL:/var OPTIONAL:/usr"

hostname=demobsd.localdomain
dhcpcd=YES
dhcpcd_flags="-qM wm0"
sshd=YES
ntpd=YES
ntpdate=YES
wscons=YES
cgd=YES

6.6 Set root password

# passwd root

6.7 Set keyboard

Set encoding <type_of_keyboard> in /etc/wscons.conf.

A full list of keyboard mappings and variants can be found in wskbd(4).

6.8 Set timezone

# ln -sf /usr/share/zoneinfo/<region/<city_or_sub-region> /etc/localtime

6.9 System locale

Example: Using en_CA.UTF-8:

# echo "export LANG=\"en_CA.UTF-8\"" >> /etc/profile
# echo "export LC_CTYPE=\"en_CA.UTF-8\"" >> /etc/profile
# echo "export LC_ALL=\"\"" >> /etc/profile

6.10 Network interface

Example: For ethernet interface wm0 that uses DHCP:

# cat > /etc/ifconfig.wm0 << EOF
up
media autoselect
EOF

7. Finish up

Exit chroot:

# exit

Unmount the new system:

# cd /
# umount /targetroot/home
# umount /targetroot/usr
# umount /targetroot/var
# umount /targetroot/

Close CGD device:

# cgdconfig -u cgd0

Reboot:

# shutdown -r now

Note: Encryption passphrase uses a us qwerty keyboard for entry regardless of setting in wscons.conf.

During boot the system halts and prompts for the passphrase to unlock the CGD drive.

Welcome to NetBSD!

8. Resources

You can like, share, or comment on this post on the Fediverse 💬

Thanks for reading! Read other posts?

» Next: Install Chimera Linux with encrypted Root-on-ZFS

« Previous: Secure remote access to NetBSD devices using SSH keys