Build a minimal Linux system and run it in QEMU

Linux is the #1 open-source operating system nowadays, and many people are running a Linux distro, such as Ubuntu or Arch. Linux is also the most popular choice for a server OS.

Building Linux from scratch is a good way to learn how it works, and is also a good practice for whoever wanting to learn about how operating systems work. And for me, the task of the first experiment of the course Operating System Concepts.

Environment setup

The lab task is to build Linux 2.6.26 and run it in QEMU. For the most convenient setup, I recommend the 32-bit versions of Ubuntu 14.04 “trusty” or Debian 7 “wheezy”. If you prefer another Linux distro, make sure it comes with glibc of a version no newer than 2.19. This is because glibc 2.20 requires Linux kernel 2.6.32, which supercedes our target version.

There’s no need to install the base system physically if you don’t have one yet, as a virtual machine will work perfectly well, and you can enjoy your Windows or Mac applications while the VM is running.

Before we start this experiment, we need to have proper tools installed. On Ubuntu 14.04 or Debian 7, the following command will install all we need for this lab:

sudo apt-get install build-essential libncurses5-dev qemu

• The build-essential package, as suggested by its name, contains essential tools for building, such as binutils, C compiler and library, and automation tools like Make.
• The libncurses5-dev package provides header files for the New Curses library, which is used to display beautiful user interface in a text terminal. Many good-looking terminal programs use it, such as Vim or Emacs.
• QEMU is what we’ll be booting our Linux with - of course it’s needed

Building the Linux system

Now we’re ready to build our own Linux. The first thing is the kernel.

Compiling the kernel

Download and extract the source code:

wget https://mirrors.edge.kernel.org/pub/linux/kernel/v2.6/linux-2.6.26.tar.gz
tar zxvf linux-2.6.26.tar.gz

Next, generate the default configuration and build against that:

cd linux-2.6.26
make i386_defconfig
make

You’ll likely encounter a few errors during the building process. Here are the fixes to two most common errors people encounter:

The first one you’ll meet should look like this:

gcc: error: elf_x86_64: No such file or directory
make[1]: *** [arch/x86/vdso/vdso.so.dbg] Error 1
make: *** [arch/x86/vdso] Error 2

To fix this, open arch/x86/vdso/Makefile in a text editor, such as Vim or gedit. Replace -m elf_x86_64 with -m64 and -m elf_i386 with -m32. Save the changes.

The second one would be like this:

undefined reference to `__mutex_lock_slowpath'
undefined reference to `__mutex_unlock_slowpath'

To fix this, open kernel/mutex.c and look for the above two functions. You’ll see them written like these:

static void noinline __sched
__mutex_lock_slowpath(atomic_t *lock_count);

static noinline void __sched __mutex_unlock_slowpath(atomic_t *lock_count);

Insert __used after the keyword static in both cases, so they should end up looking like these:`

static __used void noinline __sched
__mutex_lock_slowpath(atomic_t *lock_count);

static __used noinline void __sched __mutex_unlock_slowpath(atomic_t *lock_count);

For most people, fixing the above two things should enable the build process to complete without interrupts.

That’s the kernel. Before we can boot it up, we need an initial filesystem, with some critical files for the system to be able to boot up.

Here two options are presented. Although only one is necessary, I still recommend trying out both - for a better understanding how Linux works.

Preparing the root filesystem - Option 1: Handcraft init

With the first option, we will be creating a minimal program to serve as the “startup program”.

Open your favorite text editor and write the following C program:

#include <stdio.h>
#include <unistd.h>

int main() {
    while (1) {
        printf("Hello\n");
        sleep(1);
    }
}

Save it as test.c, and run the following command to compile it:

gcc -static -o init test.c

Now you have the init program. You need to prepare the filesystem. The following commands will create an empty 4 MB image and mount it at rootfs.

dd if=/dev/zero of=myinitrd.img bs=4M count=1
mkfs.ext3 myinitrd.img
mkdir rootfs
sudo mount -o loop myinitrd.img rootfs

Next, copy your init program into it, and create some device files as required:

cp init rootfs/
cd rootfs
mkdir dev
sudo mknod dev/ram b 1 0
sudo mknod dev/console c 5 1
cd ..
umount rootfs

After having the linux kernel and the root filesystem ready, you can try booting it in QEMU:

qemu-system-i386 -kernel linux-2.6.26/arch/x86/boot/bzImage -initrd myinitrd.img --append "root=/dev/ram init=/init"

You’ll see QEMU launching in a new window, with a lot of messages followed by the output of your init program, like this:

Preparing the root filesystem - Option 2: BusyBox

The first option is just a minimal example of what a root filesystem should contain. It is, however, not quite function-rich.

With BusyBox that packs many common Unix & Linux utilities into one single binary, you’ll be able to create a mostly functional, yet still minimal Linux system.

Busybox is available as source code so whoever need it can compile it themselves. Download the source code and configure it:

wget https://busybox.net/downloads/busybox-1.30.1.tar.bz2
tar jxvf busybox-1.30.1.tar.bz2
cd busybox-1.30.1

You need to configure some build options so it best suits this lab. Run make defconfig then make menuconfig to start. You’ll need to change at least four options as shown below. The first option is switched on and off with the space bar, and the second and the third one requires you to enter the string manually. Finally, the last one is a multiple choice. You should put the X on the desired option.

Settings –>
    Build Options
        [*] Build static binary（no share libs）

Settings –>
    (-m32 -march=i386) Additional CFLAGS
    (-m32) Additional LDFLAGS

Settings –>
    What kind of applet links to install –>
        (X) as soft-links

With the build properly configured, now you can run make then make install to build BusyBox and deploy your build. Installed files will appear under _install directory inside busybox-1.30.1.

To be able to use the _install directory as a bootable root filesystem, you should create the special files identical to what’s there in Option 1.

mkdir -p dev
sudo mknod dev/console c 5 1
sudo mknod dev/ram b 1 0

Next, you need a init program. This time we want to go the easy way with BusyBox, instead of writing a dummy one in C. Open your favorite text editor and type the following content:

#!/bin/sh
echo "### INIT SCRIPT ###"
mkdir /proc /sys /tmp
mount -t proc none /proc
mount -t sysfs none /sys
mount -t tmpfs none /tmp
echo -e "\nThis boot took $(cut -d' ' -f1 /proc/uptime) seconds\n"
exec /bin/sh

Save it with the name init under the directory _install. Run chmod a+x init so it becomes executable.

Now pack everything up as a ramdisk image. Make sure you’re inside _install directory before running the following command:

find -print0 | cpio -0oH newc | gzip -9 > ~/initramfs.cpio.gz

There should be a new file initramfs.cpio.gz in your home directory. You can now run QEMU with this new package:

qemu-system-i386 -kernel linux-2.6.26/arch/x86/boot/bzImage -initrd ~/initramfs.cpio.gz --append "root=/dev/ram init=/init"

Make sure the path to the Linux kernel is correct. Your path will likely vary depending on your procedure. You can always run find ~ -name bzImage to see where it’s located.

If everything’s going right, you’ll see the following screen in QEMU:

Congratulations! You’ve built your own Linux-from-Scratch and booted it in QEMU.

There’s a second part of the original Lab 1 of Operating System Concepts, which I will describe in a later article (or more likely, skipped :)).