Skip to content

Mic92/nix-build-uncached

Folders and files

NameName
Last commit message
Last commit date
Dec 12, 2024
Aug 27, 2020
Jul 19, 2020
Apr 25, 2020
Nov 21, 2024
Apr 26, 2020
Sep 20, 2023
Apr 22, 2023
Nov 16, 2023
Apr 25, 2020
Apr 25, 2020
Apr 22, 2023
Jan 16, 2021
Aug 25, 2020
Aug 25, 2020
Aug 25, 2020

Repository files navigation

nix-build-uncached

Deprecation nix-eval-jobs exposes the binary cache information now also in --check-cache-status, which allows to build a similar feature. nix-fast-build uses nix-eval-jobs and exposes the --skip-cached flag to replace nix-build-uncached. Otherwise NixOS/nix#3946 might track information about possible features in nix itself.

Test

nix-build by default will download already built packages, resulting in unnecessary downloads even if no package has been changed. nix-build-uncached will only build packages not yet in binary caches.

USAGE

nix-build-uncached is available in nixpkgs.

For nix versions bigger than 2.3.x you need to enable at least experimental-features = nix-command in your /etc/nix/nix.conf or $HOME/.config/nix/nix.conf or pass -build-flags "--experimental-features nix-command" to nix-build-uncached.

In the following example ci.nix contains all expressions that should be built. Since only hello-nur is not yet in the binary cache, all other packages are skipped.

[joerg@turingmachine] nix-build-uncached ci.nix
$ nix-build --dry-run ci.nix --keep-going
these derivations will be built:
  /nix/store/s5alllpjx9fmdj26mf9cmxzs3xyxjn7f-hello-2.00.tar.gz.drv
  /nix/store/03m2lwg4zia58zqm7hqlb3r0cgfq53cn-hello-2.00.drv
these paths will be fetched (198.28 MiB download, 681.37 MiB unpacked):
  /nix/store/0mijq2b50xmgk6akxdrg3x8x0k7784jb-python3.8-kiwisolver-1.2.0
  /nix/store/0q1g21q160w2cj2745r24pfn2yb8pmda-python3.8-jupyter_client-6.1.5
  /nix/store/0qxi1gzv1xgpxy58nfk9pxlfqybv1198-cntr-1.2.0
 # ...
$ nix build --keep-going /nix/store/s5alllpjx9fmdj26mf9cmxzs3xyxjn7f-hello-2.00.tar.gz.drv /nix/store/03m2lwg4zia58zqm7hqlb3r0cgfq53cn-hello-2.00.drv
[1/2 built, 1 copied (0.7 MiB)] connecting to 'ssh://Mic92@prism.r'

We can pass all arguments that also nix-build accept:

[joerg@turingmachine] nix-build-uncached -E 'with import <nixpkgs> {}; hello'
$ nix-build --dry-run -E with import <nixpkgs> {}; hello --keep-going
these paths will be fetched (0.04 MiB download, 0.20 MiB unpacked):
  /nix/store/aldyr0pjzqydf1vn9lzz7p5gvc141fhn-hello-2.10

However this only affects the evaluation during the dry build, if you want to pass arguments to the final nix build instead, use -build-flags:

[joerg@turingmachine] nix-build-uncached -build-flags '--builders ""' ci.nix
$ nix-build --dry-run ci.nix --builders
these derivations will be built:
  /nix/store/s5alllpjx9fmdj26mf9cmxzs3xyxjn7f-hello-2.00.tar.gz.drv
  /nix/store/03m2lwg4zia58zqm7hqlb3r0cgfq53cn-hello-2.00.drv
these paths will be fetched (198.28 MiB download, 681.37 MiB unpacked):
  /nix/store/0mijq2b50xmgk6akxdrg3x8x0k7784jb-python3.8-kiwisolver-1.2.0
  /nix/store/0q1g21q160w2cj2745r24pfn2yb8pmda-python3.8-jupyter_client-6.1.5
  /nix/store/0qxi1gzv1xgpxy58nfk9pxlfqybv1198-cntr-1.2.0
 # ...
$ nix build --builders   /nix/store/s5alllpjx9fmdj26mf9cmxzs3xyxjn7f-hello-2.00.tar.gz.drv /nix/store/03m2lwg4zia58zqm7hqlb3r0cgfq53cn-hello-2.00.drv
[1/2 built, 1 copied (0.7 MiB)] connecting to 'ssh://Mic92@prism.r'

Flakes

We cannot support flakes directly at the time because nix-build does not accept those and nix build's --no-dry-run is broken. However it is possible to add a wrapper nix expression that imports a flake. The following example imports hydra jobs from a nix flake the same directory. It assumes that the flake also has nixpkgs in its inputs.

let
  outputs = builtins.getFlake (toString ./.);
  pkgs = outputs.inputs.nixpkgs;
  drvs = pkgs.lib.collect pkgs.lib.isDerivation outputs.hydraJobs;
in drvs
$ nix-build-uncached ./ci.nix

Packages with allowSubstitutes = false

If your package set you are building has packages at the top level scope that have the attribute allowSubstitutes = false; set, than nix-build-uncached will build/download them everytime. This attribute is set for some builders such as writeText or writeScriptBin. A workaround is to use the following nix library and save it as force_cached.nix. Than wrap your attribute set like this:

let
  pkgs = import <nixpkgs> {};
in (pkgs.callPackage ./force_cached.nix {}) {
  hello = pkgs.writeScriptBin "hello" ''
    #!/bin/sh
    exec ${pkgs.hello}/bin/hello
  '';
}

Real-world examples