Title: How to use cpan or pip packages on Nix and NixOS
Author: Solène
Date: 18 September 2021
Tags: nixos nix perl python
Description:
# Introduction
When using Nix/NixOS and requiring some development libraries available
in pip (for python) or cpan (for perl) but not available as package, it
can be extremely complicated to get those on your system because the
usual way won't work.
# Nix-shell
The command nix-shell will be our friend here, we will define a new
environment in which we will have to create the package for the
libraries we need. If you really think this library is useful, it may
be time to contribute to nixpkgs so everyone can enjoy it :)
The simple way to invoke nix-shell is to use packages, for example the
command ` nix-shell -p python38Packages.pyyaml` will give you access to
the python library pyyaml for Python 3.8 as long as you run python from
this current shell.
The same way for Perl, we can start a shell with some packages
available for databases access, multiples packages can be passed to
"nix-shell -p" like this: `nix-shell -p perl532Packages.DBI
perl532Packages.DBDSQLite`.
# Defining a nix-shell
Reading the explanations found on a blog and help received on Mastodon,
I've been able to understand how to use a simple nix-shell definition
file to declare new cpan or pip packages.
HTML Mattia Gheda's blog: Introduction to nix-shell
HTML Mastodon toot from @cryptix@social.coop how to declare a python package on the fly
What we want is to create a file that will define the state of the
shell, it will contain new packages needed but also the list of
packages.
# Skeleton
Create a file with the nix extension (or really, whatever the file name
you want), special file name "shell.nix" will be automatically picked
up when using "nix-shell" instead of passing the file name as
parameter.
```nix configuration file
with (import <nixpkgs> {});
let
# we will declare new packages here
in
mkShell {
buildInputs = [ ]; # we will declare package list here
}
```
Now we will see how to declare a python or perl library.
## Python
For python, we need to know the package name on pypi.org and its
version. Reusing the previous template, the code would look like this
for the package Crossplane
```nix configuration file
with (import <nixpkgs> {}).pkgs;
let
crossplane = python37.pkgs.buildPythonPackage rec {
pname = "crossplane";
version = "0.5.7";
src = python37.pkgs.fetchPypi {
inherit pname version;
sha256 = "a3d3ee1776bcccebf7a58cefeb365775374ab38bd544408117717ccd9f264f60";
};
meta = { };
};
in
mkShell {
buildInputs = [ crossplane python37 ];
}
```
If you need another library, replace crossplane variable name but also
pname value by the new name, don't forget to update that name in
buildInputs at the end of the file. Use the correct version value too.
There are two references to python37 here, this implies we need python
3.7, adapt to the version you want.
The only tricky part is the sha256 value, the only way I found to find
it easily is the following.
1. declare the package with a random sha256 value (like echo hello |
sha256)
2. run nix-shell on the file, see it complaining about the wrong
checksum
3. get the url of the file, download it and run sha256 on it
4. update the file with the new value
## Perl
For perl, it is required to use a script available in the official git
repository when packages are made. We will only download the latest
checkout because it's quite huge.
In this example I will generate a package for Data::Traverse.
```shell instructions
$ git clone --depth 1 https://github.com/nixos/nixpkgs
$ cd nixpkgs/maintainers/scripts
$ nix-shell -p perlPackages.{CPANPLUS,perl,GetoptLongDescriptive,LogLog4perl,Readonly}
$ ./nix-generate-from-cpan.pl Data::Traverse
attribute name: DataTraverse
module: Data::Traverse
version: 0.03
package: Data-Traverse-0.03.tar.gz (Data-Traverse-0.03, DataTraverse)
path: authors/id/F/FR/FRIEDO
downloaded to: /home/solene/.cpanplus/authors/id/F/FR/FRIEDO/Data-Traverse-0.03.tar.gz
sha-256: dd992ad968bcf698acf9fd397601ef23d73c59068a6227ba5d3055fd186af16f
unpacked to: /home/solene/.cpanplus/5.34.0/build/EB15LXwI8e/Data-Traverse-0.03
runtime deps:
build deps:
description: Unknown
license: unknown
License 'unknown' is ambiguous, please verify
RSS feed: https://metacpan.org/feed/distribution/Data-Traverse
===
DataTraverse = buildPerlPackage {
pname = "Data-Traverse";
version = "0.03";
src = fetchurl {
url = "mirror://cpan/authors/id/F/FR/FRIEDO/Data-Traverse-0.03.tar.gz";
sha256 = "dd992ad968bcf698acf9fd397601ef23d73c59068a6227ba5d3055fd186af16f";
};
meta = {
};
};
```
We will only reuse the part after the ===, this is nix code that
defines a package named DataTraverse.
The shell definition will look like this:
```nix shell definition code
with (import <nixpkgs> {});
let
DataTraverse = buildPerlPackage {
pname = "Data-Traverse";
version = "0.03";
src = fetchurl {
url = "mirror://cpan/authors/id/F/FR/FRIEDO/Data-Traverse-0.03.tar.gz";
sha256 = "dd992ad968bcf698acf9fd397601ef23d73c59068a6227ba5d3055fd186af16f";
};
meta = { };
};
in
mkShell {
buildInputs = [ DataTraverse perl ];
# putting perl here is only required when not using NixOS, this tell you want Nix perl binary
}
```
Then, run "nix-shell myfile.nix" and run you perl script using
Data::Traverse, it should work!
# Conclusion
Using not packaged libraries is not that bad once you understand the
logic of declaring it properly as a new package that you keep locally
and then hook it to your current shell session.
Finding the syntax, the logic and the method when you are not a Nix
guru made me despair. I've been struggling a lot with this, trying to
install from cpan or pip (even if it wouldn't work after next update of
my system and I didn't even got it to work.