Chapter 5 - How Did We Get Here

Now it’s time to go back to basics, this chapter’s going to explain how Nix works internally and why it works. It’s good to have knowledge of this, especially when troubleshooting.

In this chapter we will learn…

1. The fundamental principles of Nix
2. How a Nix package is made
3. How the Nix store is edited and used
4. How NixOS plays into all of this
5. Implications of this paradigm
6. What a flake is
7. How to collect garbage

Nix Fundamentals

Nix is based on functional programming, which values purity and reproducibility. What this means for Nix is it sees every package as a result of a pure function, that is, a function with no side effects.

A pure function will always output the same thing given the same inputs. Nix packages should always build the same given the same inputs.

Unlike other Linux package managers, Nix won’t use currently installed libraries/tools while building. All dependencies of a Nix package must be declared. This ensures that the package will always build the same on any machine.

How a Nix Package is Made

Nix packages are represented by a Nix expression, similar to how our NixOS config is an expression.

package.nix

# Basic (and probably not exactly correct) nix package expression
{
    rustPlatform,
    gtk,
    ...
}: rustPlatform.buildRustPackage {
    src = ./.; # Assuming this is in a flake, otherwise we'd need to use a function to fetch source from GitHub, etc
    pname = "my-package";
    version = "0.1.0";
    cargoLock.lockFile = ./Cargo.lock; # We need to include lock files because Nix needs to know *exactly* which versions of each dependency to fetch

    buildInputs = [ gtk ];
}

Note

Like a NixOS module, we can define packages inline or in a separate file as a function. Most people prefer to do it as a separate file for organization.

Nix evaluates a package expression and creates a derivation from it.

This derivation is then placed in the Nix store for use later.

• Directory/nix/store/

  • [hash]-my-package-0.1.0.drv Instructions on how to build my-package
  • …

This contains everything the Nix build system will need to build the package: which other packages are needed, what commands to run, environment variables to set, etc.

How the Nix Store is Edited and Used

After building a package, the output of the build is placed in the nix store, this is a folder usually located in /nix/store.

You can check on both your host system and VM for the existence of this directory and see it has many items. You’ll also notice the directories all have a hash after the name, this is so no conflicts exist when building two versions of the same package.

• Directory/nix/store/

  • [hash]-my-package-0.1.0.drv Instructions on how to build my-package
  • [hash]-my-package-0.1.0 Output of my-package
  • …

Everything nix builds is put in the nix store. When we do nix shell, the package is built and placed in the nix store, and then the path to the bin folder of the package’s output within the nix store is placed within the PATH environment variable. You may also notice that result in your config’s directory is a symlink to within the store.

The nix store is the single source of truth, everything else (even many things within the store itself) reference or symlink to it.

Note On Dynamically Linked Binaries

This does pose a bit of an issue with programs that are dynamically linked against system libraries. As these libraries are in the nix store and not where the binary expects, they will fail to run. Nix has the ability to build minimal containers that conform to how binaries expect the system to be laid out (called an FHS - Filesystem Hierarchy Standard).

Any programs built with Nix itself are patched to instead link against the proper paths in the nix store, this issue only comes up with binaries downloaded from the web (some npm packages may do this!).

How NixOS works

NixOS takes Nix a step further and stores system files within the store. This can mostly be seen in /etc in your VM. Notice how many symlinks there are that head to the nix store. Notice how your shell’s default PATH is set to include the nix store (or /run/wrappers/bin which links to it eventually as well). NixOS achieves a reproducible system by using the store as a single source of truth.

This also makes switching your NixOS config easy. Switching is really just a matter of changing what symlinks point to as well as updating environment variables (there are other things NixOS does as well, especially for hardware and bootloaders, but the point is it knows how to perform those operations based on your config).

Note

This diagram is a simplification, switching /run/current-system is only one part of switching to a new config.

NixOS also needs to update other paths, start/stop systemd services, potentially install a new bootloader, etc.

With NixOS, stuff like your /etc folder is effectively a package, and therefore we maintain that it must be able to be reproduced the exact same given the same set of inputs (your configuration). Because we build them from the ground up every time, there’s no way for a unknown side effect to taint the system’s functionality.

When we make a change to our config we don’t need to rebuild everything of course. Nix is smart in that it knows what has and hasn’t changed (because all packages are pure). So when we make one small change in, say, something that affects our /etc/passwd, we don’t need to rebuild everything, because the dependencies of /etc never changed (except passwd), we can simply relink them the exact same.

Amazing Implications

These rules set forth by Nix and NixOS provide us with some great features.

Binary Caches

Binary caches are why we don’t need to recompile Firefox or Plasma from scratch when we add them to our config. If a package exists in the remote cache (by default Nix has cache.nixos.org as the only one), we can simply download its output (bundled in a NAR file). And we’re sure the output will be the same because the package is pure.

cache.nixos.org isn’t the only binary cache too! In fact, it’s very easy to make your own system serve its nix store using the nix-serve command.

nix-serve --host * --port 8080

This will allow other computers on the network to use your nix store to substitute packages if needed. Obviously there are security risks that can come with using a cache without knowing its reliability, Nix provides mechanisms to verify cache’s identity to ensure trust.

Personally I have two devices that use pretty similar NixOS configs, with many of the differences only being in hardware. By first building on my better computer and then hosting a cache off it, I not only don’t have to build many packages on my other computer, I can also download any packages I would normally use cache.nixos.org for, which will be way faster over LAN!

Remote Builders

Another advantage is we can use other computers to build Nix packages effortlessly. Because, again, packages are pure we know that a package built on one system will work on another. We can use this guarantee to have other computers (maybe more powerful computers) build for us.

Effortless VMs

You may have noticed that our nixos.qcow2 file is relatively small. That’s because NixOS doesn’t simply copy the Nix store into the image, it actually uses QEMU’s virtio feature to expose the host system’s nix store directly to the VM. This means we don’t have to rebuild as much if we delete nixos.qcow2, and also we don’t have to create a huge file system image for our VM.

From a higher level, NixOS is also great for VMs because it can reliably produce images exactly how we want. I’ve used NixOS VMs many time to perform end-to-end tests on my software.

The versatility of NixOS with VMs is so good that nixpkgs actually uses it to test every single package by running a variety of tests within a virtual machine in CI.

ISOs

You can also build an ISO of a given NixOS system configuration (provided it’s setup a certain way). We’ll actually use this next chapter to generate a personalized installer image just for you.

Docker Containers

Nix also allows the creation of docker containers. Many people compare Nix and Docker for reproducible builds. In my opinion Nix should be used to create an image, but Docker should still run it (because Nix can’t obviously). With nix you have the full versatility of a functional language and nixpkgs (the largest package repository) to build images.

Reliable CI

Writing CI scripts can get challenging, especially when there’s no way to truly test if the script will work given that you can’t emulate the CI’s environment (at least not easily, specifically with GitHub Actions). With Nix, you can create packages and checks declaratively and use pre-built actions to build/run them, and they’ll always be in the correct environment!

No More Guides

With NixOS, the intent and effect of a config is right there. No more guides saying “go here and edit this random rc file” or blah blah. Many times when you want to do something it’s as simple as looking at /stealing someone else’s configuration. I personally use SourceGraph (tuned to Nix) in order to find how other people configure things. There’s also the NixOS option search to help you.

When someone asks how you did something in NixOS you can simply share the configuration snippet that does it.

Time Travel

Sure many backup solutions exist for non-NixOS systems. But what those should really be backing up is your personal data. Your system files would go untouched. With nix, your config is a git repo, you can rollback, bisect, diff, etc. all previous versions of your system like it’s just another programming project. This also means you can publish your config to GitHub or another cloud hosting service to keep it safe.

When you install you’ll also see that after booting you’re given a list of previous system generations to boot into, so if you manage to make your current system unbootable, you can always rollback to a previous config. Storage-wise this works because there are many similarities between your old and current systems and therefore they only need to take up storage space for what’s different.

Write Once, Configured Always

Once you figure out how to do something with Nix, the chances it needs to be done again are slim (NixOS can always update to rename options, etc.). And with Git you never lose these changes so long as you commit. They will always be there in your repo history ready to grab again just in case.

Flakes

Flakes were a controversial feature, before them we used channels. This guide is written to purely use flakes as they’ve been proven as an effective way to share various parts of the Nix ecosystem.

A flake is really just any flake.nix file in a folder or at the root of a git repository that can represent anything. When we run nix run or nix shell we’re expected to pass flake#output, where flake can be a . for the current directory, github:Author/Repo for a github repo, or nixpkgs for nixpkgs. We can also configure custom aliases for flakes. We then pass the output we want. Depending on context we can omit packages. or nixosConfigurations..

Flakes define their dependencies based on inputs, which are links to other flakes to fetch in order to evaluate the current flake properly. The hashes of these other flakes are stored in flake.lock so we can make sure all the inputs are exactly what we expect.

Before flakes we had other ways of using Nix, but for this guide we’ll stick to just them.

Garbage Collection

So, we know that the nix store holds everything. But how do we know when we don’t need an entry in the store anymore?

Nix has a special folder, /nix/var/nix/gcroots that holds many symlinks to the nix store.

These symlinks represent gcroots. When we want to clean up our system, we look at these roots. Any store entry that isn’t a descendant of them (remember how we know what entries depend on what other entries because we have to declare all dependencies), and delete them. This way we only delete anything that isn’t being used actively.

So, when comma downloads a command for us, the store entry that package is in (and any dependencies it references) are not a descendent of a gcroot. Meaning when we garbage collect after running comma, the store entry we added and all of its dependencies (that aren’t being used by something else that is under gcroots), will be deleted.

Note

“Oblivion” here just means deleting the directory. My diagram library doesn’t have a trash can shape :P.

But what creates gcroots?

• Current NixOS config (/nix/var/nix/gcroots/current-system)
• The NixOS config We Booted With (/nix/var/nix/gcroots/booted-system)
• Anything we installed with nix profile install (/nix/var/nix/gcroots/per-user)
• result symlinks from project directories (like our VM!) (/nix/var/nix/gcroots/auto)
• Processes can hold roots as well (this is how nix shell works)
• and more…

Now You Know

Thank you for hearing my spiel. We’re finally gonna install NixOS in the next chapter. Well… prepare our config to install it.