Nix for Python, attempt number 2

Last post we tried to make a Python project fully reproducible based on a detailed requirements.txt file. We ended up not managing to go the full Nix way, but we got pretty close. However, there are many ways to approach this, with language-specific tools getting published and archived on the regular.

At the end of the last post I mentioned dream2nix and their “Build your Python project in 10 minutes” tutorial which I would try out in a future post. This is that post!

Like last time, the dream2nix team have prepared a flake template so that in our project we can just run the following to initialize our project flake:

1nix flake init -t 'github:nix-community/dream2nix#simple'

Side note: The single quotes are my addition. I don’t know why, but the Nix tutorials I’ve seen don’t put in the quotes, so that my shell (zsh) complains:

1zsh: no matches found: github:nix-community/dream2nix#simple

Here’s what the flake.nix file looks like:

 1{
 2  inputs.dream2nix.url = "github:nix-community/dream2nix";
 3  outputs = inp:
 4    inp.dream2nix.lib.makeFlakeOutputs {
 5      systemsFromFile = ./nix_systems;
 6      config.projectRoot = ./.;
 7      source = ./.;
 8      projects = ./projects.toml;
 9    };
10}

It has one input flake, dream2nix. As for the outputs, dream2nix does stuff so that we only have to focus on a few options. In particular, it assumes that a couple of files are present: nix_systems and projects.toml, in the root of the project.

As the tutorial states, nix_systems lists all the systems that this flake should work on; it looks like this:

1x86_64-linux
2x86_64-darwin

However, I think I prefer listing the systems directly in flake.nix because I don’t like having another tiny file in the project root if I can help it. Following the dream2nix tutorial I can modify flake.nix like so:

 1{
 2  inputs.dream2nix.url = "github:nix-community/dream2nix";
 3  outputs = inp:
 4    inp.dream2nix.lib.makeFlakeOutputs {
 5      systems = [ "x86_64-darwin" ]; # I'm on MacOS
 6      config.projectRoot = ./.;
 7      source = ./.;
 8      projects = ./projects.toml;
 9    };
10}

It’s also possible to make use of the flake-utils library to simplify this part, but I’ll cover that some other time.

The projects.toml file contains extra information about the project, but the tutorial does not expand on this; supposedly because the contents are highly dependent on the kind of project (programming language, package manager…). We can run an auto-detection program provided by dream2nix (note the single quotes):

1nix run 'github:nix-community/dream2nix#detect-projects' . > projects.toml

In our case, projects.toml looks like:

 1# projects.toml file describing inputs for dream2nix
 2#
 3# To re-generate this file, run:
 4#   nix run .#detect-projects $source
 5# ... where `$source` points to the source of your project.
 6#
 7# If the local flake is unavailable, alternatively execute the app from the
 8# upstream dream2nix flake:
 9#   nix run github:nix-community/dream2nix#detect-projects $source
10
11[main]
12name = "main"
13relPath = ""
14subsystem = "python"
15translator = "pip-freeze"
16translators = [ "pip-freeze",]

Here, some problems start. First, I don’t understand what exactly this content means. Here is what I think is happening though: I think dream2nix has understood that this is a Python project, and it’s understood that the requirements are declared with the pip freeze command, which sounds reasonable. However, the docs are not quite explicit enough for me to understand if I need to do anything else at this point. Let’s assume everything is fine and dandy and carry on. In fact, following the next step of the tutorial, I can see what Nix sees:

 1$ nix flake show
 2warning: Git tree '/Users/Auguste/Desktop/contact-app' is dirty
 3git+file:///Users/Auguste/Desktop/contact-app
 4├───apps
 5│   └───x86_64-darwin
 6│       └───detect-projects: app
 7├───devShells
 8│   └───x86_64-darwin
 9│       └───default: development environment 'nix-shell'
10└───packages
11    └───x86_64-darwin
12        ├───default: package 'python3.10-default'
13        └───resolveImpure: package 'resolve'

There is in fact a Python package here, and a devShell that I could supposedly drop into that has all the dependencies available.

Let’s try this! According to the tutorial, we can build the project like so:

1$ nix build '.#default'
2warning: Git tree '/Users/Auguste/Desktop/contact-app' is dirty
3error: getting status of '/nix/store/hharq96d2b9jni4scs4mxw573hsfx20b-source/requirements-dev.txt': No such file or directory
4(use '--show-trace' to show detailed location information)

This fails and complains that there is no requirements-dev.txt file to be found. There is indeed none in the project, although it’s not supposed to be mandatory. Strange… Let’s try again, having added an empty requirements-dev.txt in the root of our project.

1$ touch requirements-dev.txt
2$ nix build '.#default'
3warning: Git tree '/Users/Auguste/Desktop/contact-app' is dirty
4error: getting status of '/nix/store/hharq96d2b9jni4scs4mxw573hsfx20b-source/requirements-dev.txt': No such file or directory
5(use '--show-trace' to show detailed location information)

Well, that changed nothing.

I decided to go ahead and ask the maintainters about this directly: maybe I missed some obvious step in the process. Tune in soon to see if I get an answer!

This is post number 002 of #100daystooffload.