💾 Archived View for dvejmz.srht.site › 2016-08-23-compiling-linux-kernel.gmi captured on 2023-01-29 at 15:49:30. Gemini links have been rewritten to link to archived content
⬅️ Previous capture (2022-03-01)
-=-=-=-=-=-=-
The Linux kernel was written by Linus Torvalds, a Finnish student of the University of Helsinki. Torvalds started working on a new kernel, drawing inspiration from MINIX, a minimalistic Unix variant very popular at the time, especially in academia.
The first release of his personal project, a tiny working kernel, was made in 1991. Soon after a number of volunteers joined Linus in what they'd soon realise was bound to become something much bigger than a simple MINIX derivative. Nowadays, Linux, at its 4.7.2 (stable) revision has become one of the most impressive projects ever attained by mankind, and that is not an overstatement. With well over 7 millions lines of code, improved and extended every day by thousands of volunteers and organisations across the globe, Linux is the biggest software project ever created. It is also the most ubiquitous OS, running on most of the server and embedded space, including mobile (Android). You may be aware of it or not, but chances are there will be at least one device in your house running some version of Linux: a desktop computer, a smartphone, a set-top box or even a toaster. Linux can handle anything.
As it is usual with open source software, if you have special requirements for a particular application, you can compile it yourself and override most of its configuration to obtain something that works in your case. Kernels are no exceptions to this. There are a number of reasons why you'd want to build the kernel running your computer yourself. Most of Linux's source code is licensed under a (L)GPLv2 license, which means it's almost always safe to modify the code and use it however you please. Linux maintainers do encourage people to modify the kernel to suit their particular needs but doing so is a daunting endeavour. I mean, you don't just download some 7 million lines of code and reasonably expect to make some tweaks here and there as though you were just fixing your toaster.
Fortunately, you don't need to possess godlike C programming skills and an amount of knowledge about the innards of the kernel that can only be acquired with years of experience to build support into Linux for some obscure device you might be using at home or some niche filesystem you need to use for whatever reason. If you at least have a reasonably clear idea of what you want to achieve and the implications, adjusting Linux to match that is not a particularly hard task, and I'm going to give you a brief outline of how you can go about it.
The first thing you want to do is download the sources for the version of Linux you wish to build from
. Most of the time, this will be the latest stable release but you can download the sources for the latest mainline release candidate (a more cutting-edge version) instead if you're feeling adventurous. Once downloaded, unpack the contents of the TAR file into a directory of your choosing.
$ tar -xf linux-.tar.xz
As any other program written in C, the Linux kernel is built using the `make` program. `make` usual targets are things such as `cleanup` or `test`, and if run with no arguments, it will build and link the entire application. However, Linux defines a number of special targets of its own to control specifically how it's configured. These targets will run in turn scripts that will invoke a number of different configuration frontends. These frontends will output your preferred settings to the master `.config` file that will be used by `make` to build the kernel how you intend it to.
After downloading and extracting, we need to clean up the source tree with `./mrproper` to lay the groundwork for the compilation step. Once we've done that, we can jump right into configuring the kernel itself, which is the most vital step of this process and the chief reason why the Linux kernel is manually compiled. But first we need to learn how to access this configuration.
`make` makes use of a master configuration file to store all the flags to compile the Linux kernel. This file is called .config and does not come with the source published on kernel.org. You need to create it yourself. It makes sense if you think about it. How could the kernel maintainers distribute a default `.config` file with some allegedly sane preset options when there are so many different platforms and configurations it may be built for? Including support for 64-bit, Realtek network cards and multithreading may seem like some pretty sane defaults for an average modern laptop Linux kernel but they would not make any sense in a router controller. But fret not: the fact that you're not given a template straight away to get a headstart into your kernel configuration doesn't mean you're not given a default way to get it. Turns out you can generate an empty config file template by running `make config` or a preset one with `make defconfig`, although the easiest way to get it would be to get a copy of your own distro kernel config file from /proc/config.gz (Arch Linux) or /boot (Debian).
Run `make menuconfig` to run the basic terminal-based menu to customise the options you want. There are many other config front-ends available: nconfig (ncurses), xconfig (Qt) and gconfig (GTK).
Execute `make help` if you want to get a full listing of all the possible options you can pass to it.
https://s3-eu-west-1.amazonaws.com/sgfault-b01/assets/img/2016/8/kernel-2.png
Finding your way through the different options in the terminal frontends, namely _menuconfig_ and _nconfig_ is easy enough: you'll be presented with a list of kernel flags, known as symbols, organised in a category tree. Flags are very well organised under very clear category groups. You can start your way from the main panel and select the categories that pique your interest to drill down into more specific subcategories and flags. Press `<SPACE>` to toggle Yes/No symbols, `<ENTER>` for symbols which expect text (enclosed between parentheses), `<F2>` to obtain detailed information about a specific symbol and `<F8>` to search for one by keyword. To exit, press `<F9>`. You'll be prompted to save your changes to the `.config` file. The GUI counterparts work just the same and don't require any further explanation. All frontends perform the same essential task: choosing one is purely a matter of taste.
At this point, you have a lot of options (and I mean *a lot*) stacked in front of you: device drivers, filesystems, networking, kernel execution policies... the list goes on and on. What are they for? Which ones do you need? Which ones do you _not_ need? It'd be quite impractical -and pointless- to attempt to explain all the different categories you'll encounter in this file and their function. I encourage you to spend some time discovering them on your own to get the gist of what it's available. Every single flag is documented. You can access the description for a flag with the SymInfo option (`<F2>`) in the terminal configuration frontends; these descriptions will appear straight away in the Qt and GTK clients. If you already have a specific constraint in mind you'd like to override, use the symbol search option (`<F8>`/CTRL+F).
As an example, we're going to show how to override a pretty basic kernel option. Say you're interested in changing the local version name of your custom kernel so that it's easier to tell apart from any others installed in your system. This is a string which gets appended to the version of the kernel. In Arch Linux, this label is usually "-ARCH" but it will vary wildly between distros. This is both displayed in GRUB's kernel selection menu and when issuing the `uname -r` command in a shell.
$ uname -r
$ 4.2.5-1-CUSTKERNEL
To find this option, we can perform a simple symbol search (by pressing `<F8>` in the terminal frontends or CTRL+F in the GUI ones) and enter the keyword _VERSION_. Doing this will show a list of flags whose name contains the keyword specified, with their corresponding location in the flags tree. The search results reveal that the flag we are looking for is called `LOCALVERSION` and is located under _General setup_ ->` _Local version - append to kernel release_. You can then navigate to this path from the root location and set the option to any value you wish.
https://s3-eu-west-1.amazonaws.com/sgfault-b01/assets/img/2016/8/kernel-3.png
https://s3-eu-west-1.amazonaws.com/sgfault-b01/assets/img/2016/8/kernel-4.png
When you're happy with the flags you've provided, we can move on and compile the kernel with `make`. Some flags you might be interested in passing to it are,
These flags are completely optional. Invoking _make_ with no targets will do the job just fine, only noticeably slower than it would do with more threads in a multi-core computer. I am going to build the image on the same directory as the source, so we can just do
$ make -j16
The compilation took 1 hour on my 2nd Gen Intel Core i7-2630QM; this is a really CPU-intensive operation but it barely uses memory.
The first time you compile a kernel, my recommendation is to go small. Kernels are extremely complex and intricate works of engineering. If you try to disable or specify too many custom options, chances are your kernel will become unstable if it boots at all. The easiest way to go about this is to copy the kernel config file of your Linux distribution without alterations and booting your custom kernel on the same distro as the one you're using to compile it. For example, if you're compiling the kernel on Fedora, load it on a Fedora box as well instead of trying to launch a Debian or openSUSE installation with it. If all went OK, you can start flipping some options here and there, rebuild the kernel and see the effect it has on your system. In other words, sanity test your build environment by recompiling the same kernel you currently use to make sure there's nothing wrong with it. After you've made sure you can build a kernel tailored for your specific distro and hardware setup, you can start experimenting.
We should now have a -hopefully- working image of the Linux kernel, namely _vmlinux_, ready to go. This file may be different from your distribution-provided stock image in a number of ways. First, the name of your distro's image will probably be _vmlinuz_ (or even _vmlinuz-linux_, as it is in my Arch install) instead of _vmlinux_. Second, your custom image will probably be somewhat larger too (mine is about 16MB whereas the stock one is nearly 4MB). The reason for both of these discrepancies the fact that the stock _vmlinuz_ image is compressed, hence, the trailing 'z' character which usually implies compression in Unix terminology. We do not need to compress our custom image to run it though so we can install it straight away.
Installing a Linux kernel normally means placing its external components (modules) in the `/lib/modules` folder and the executable itself in `/boot` so that the bootloader (normally GRUB) can find it and load it from the filesystem when starting up the machine. Probably the first few times you compile a custom kernel, it won't deviate a great deal from your distro's defaults, so there's not much to worry about, but when you start effecting more impactful changes onto your custom kernel, you may or may not need to install an initramfs _initrd_ file. The initial RAM filesystem is an ephemeral userland filesystem the kernel utilises to load some essential modules such as disk drivers so that it can mount the actual root filesystem and carry on booting up the system. Either way, there's no reason why you couldn't use your own distro's _initrd_ file. If your distro doesn't supply it though, you can produce it with tools such as _mkinitcpio_.
Most distributions should include a package called `mkinitrd` which contains the `installkernel` scripts. These scripts must be run with root privileges and will be used to install the kernel. First, we need to install the kernel modules, if we have any:
# make modules_install
The kernel itself is installed using the common `make install` target, which is used for most software built from source:
# make install
Verify all the files belonging to your custom kernel are located in their expected places. Run `update-grub` if the make scripts didn't notify GRUB of this new kernel so that it appears on the list when you restart the computer.
Beware GRUB will always attempt to boot from the most recent kernel, whether it works or not, so if you find your newly built custom kernel is misbehaving and your distro hides the bootloader menu during bootup, hold `<SHIFT>` after starting your computer until the menu appears. From here, you'll be able to pass additional flags to load or debug your kernel or just boot from the kernel installed by your distro.
In reality, if you're an average desktop or server Linux user, you won't find yourself building a custom Linux kernel too often. This makes more sense in the embedded space or if you have really specific needs. Of course, if you belong to either of the formerly mentioned groups of users and you are also obsessed with clutter and performance, you may still wish to turn off support for devices or filesystems which may be built into the kernel by your distribution by default for some reason. Things such as support for filesystems you may never use like VFAT, NTFS (if you're never going to dual-boot), HFS, etc. Or maybe you're not interested at all in building a WiFi or Bluetooth controller in a server or desktop.
If you find that this is actually quite aligned to your preferences, you may consider switching to Gentoo (or Funtoo). Building the kernel yourself is one of the first things you'll have to do when installing them.
If you're upgrading to a newer custom kernel, you may want to remove the older one (this is something your distribution upgrading software takes care of automatically). You'll find uninstalling a kernel is easier than installing it. All we need to do is delete all of its module and its image from their installation folders and notifying the bootloader that we'll no longer load this kernel.
Delete all the files from `/boot`:
$ sudo rm -i /boot/*<kernel-version>*
Delete all the installed modules from `/lib/modules`:
$ sudo rm -i /lib/modules/*<kernel-version>*
Where `kernel-version` is the version of the kernel you installed from source.
Finally, update GRUB so the entries are removed from the boot-up menu:
$ update-grub
Linux Kernel in a Nutshell, by Greg Koah-Hartman.
How Linux Works, What Every Superuser Should Know, by Brian Ward.
https://wiki.archlinux.org/index.php/Kernels/Compilation/Traditional
http://community.linuxmint.com/tutorial/view/1718
