As some of you may know I have been a Gentoo user for many years. Although you can create and use binary caches, in Gentoo the primary way of installing software packages is by compiling them at the spot. In this post I will try to explain some of the advantages of this and how the philosophy permeating this design choice has influenced my approach to reporting issues.
I think the first thing I should do is be clear with you. Compiling software takes time and resources. The larger the software, the more resources needed. Sure, faster processors, larger caches and RAMs and technologies like SSDs are significantly reducing these, but still, large packages like libreoffice can take a while to compile on a fast modern system (and even more in older ones). And what can you do meanwhile? Well here’s a suggestion from Randall Munroe:
Now that we are over with the mandatory XKCD reference let’s try to focus on what this approach usually allows you to do. On one hand with this approach you can choose the toolchain, i.e. the tools you use for compiling. This is what allows projects like Gentoo Hardened striving to create systems more resilient against attacks to exist. Also, you can influence the options and features that the compiler uses so that they will be better suited to your machine and increase performance. And last, but not least, you can choose the compile time features you want the package to use. But, there is one more thing: you can also patch the software before compilation to improve it and solve issues.
It’s quite clear that this kind of features has as main target knowledgeable users who like to control the small details of the software running on their system and knowing what is under the hood. This target includes, but is not limited to, people in hacker, maker and DIY communities.
So what makes Gentoo different from Linux From Scratch (LFS)? In LFS the user is expected to manually run all the compilation steps by hand whilst, in Gentoo you can use a tool that (with the notable exception of the Linux kernel) will perform these steps for you based on your configuration: portage. This lowers entry barriers a bit for new users and makes automatic package upgrades a bit easier. Keep in mind though, that with good documentation everybody can run either of these systems.
So how does Gentoo automate the compilation steps? It uses a lot of tricks, but the core of any Gentoo software package is a type of file called ebuild. Ebuilds are a set of text files in your Gentoo installation (one per package and version). These files, give instructions on where to fetch a package’s sources, which are its dependencies, which patches to apply, how to map user choices to specific compilation time parameters and how to build and install the package. For most software the amount of code needed inside an ebuild is quite small as in most cases sane defaults are provided or can be loaded from auxiliary files. Ebuilds are usually accompanied by other auxiliary files which include things like software patches, metadata, external files to be provided, and even hashes for verification.
This may hint that, if you want to change how the package is compiled a solution may be to copy the ebuild and modify it yourself, this is probably not the simplest approach but if you want to do so you should consider a bit on how to set up your own “overlay” or personal ebuild repository. If you want to make core modifications to the build process, for example by adding new compile time options, this is likely the way to go though.
In most cases, this is not necessary though. You can use the tool called ebuild in order to run a packages compilation steps for fast patch writing. Running ebuild "package.ebuild" prepare will leave the software with all of the patches applied on the temporary directory (usually a folder inside /var/tmp/portage) waiting for you to apply any additional patches. To try to compile and install the package you can use ebuild "package.ebuild" merge. Although in most cases you may want to stop at an earlier stage to test the software first. You can check ebuild‘s man page for a list of the available steps at which you can stop. In most cases, earlier steps will be performed for you. Make sure to always run the clean step when you are done to remove any left over temporary files. Also avoid using this for anything other than debugging and agile patching as it can produce strange errors, for example by installing software without ensuring it’s run time dependencies are satisfied.
Obviously manually patching the software is an unsustainable approach for anything other than fast tests of software, and you probably don’t want to setup your own overlay for a more long term approach, because of this most ebuilds allow users to provide custom patches. You can find out the ebuild file’s EAPI by checking for that variable in the file, if it is missing then it is 0. EAPI indicates which version of the default behavior and available features will be used.
By now you should have gotten an idea about how easy writing and testing patches is in Gentoo. No complicated external tools like quilt or the likes, just modify the software locally or with a custom applied patch file and enjoy.
You may be wondering what has all of this to do with reporting issues (feature requests, bugs, security problems, you name it). The answer is that the user no longer needs to be a passive entity anymore. Instead, knowledgeable users can look for (or write) a patch that solves their use case, keep it applied on their systems and request the person maintaining the software package and the developers of the software themselves to consider including this patch in their next release.
This power to do and test modifications easily is a core enabler of my philosophy when reporting issues. I believe that enabling the developer so your issue can be fixed is critical. Therefore, instead of expecting the other side to perform all of the work I try to facilitate their work as much as possible. In practical terms this means that I try my best to go beyond a good description of the issue and try to also include a patch or clear instructions on how to address the problem. The final result is that the issues are more likely to be addressed and if they aren’t, well I still have my system patched anyways.
In my next post on this topic I will describe how this philosophy can be applied from a security manager’s or auditor’s perspective to increase the overall security of a project or organization.