nixpkgs_cc_configure_hermetic

This introduces a new repository rule `nixpkgs_cc_configure_hermetic`
which is meant to replace `nixpkgs_cc_configure`. The latter is
implemented by calling Bazel's builtin CC toolchain autodetection with
the binary paths overriden to point to Nix provided tools. This is more
hermetic than Bazel's default behavior in that the tools are not just
picked from `$PATH`. However, it is not hermetic. Bazel's CC toolchain
autodetection is influenced by environment variables, and it shells out
to the provided `cc` binary in the context of a repository rule to
determine system include paths which is also affected by environment
variables. Concretely, this caused cache misses due to include paths
provided by a `nix-shell` changing the cache keys of build actions
depending on the CC toolchain.

`nixpkgs_cc_configure_hermetic` on the other hand uses binaries
determined by a Nix derivation and performs additional configuration
such as determining system include paths within a Nix build, i.e. within
Nix's own sandbox. The results are stored in a text file
`CC_TOOLCHAIN_INFO` which is then parsed in a Bazel repository rule.
From there on the toolchain configuration follows the example of Bazel's
builtin CC toolchain for Unix:
https://github.com/bazelbuild/bazel/blob/0f4c498a270f05b3896d57055b6489e824821eda/tools/cpp/unix_cc_configure.bzl#L310.

Users can override the Nix provided compiler using the `nix_file` or
`nix_file_content` and `attribute_path` arguments. The corresponding Nix
derivation should produce the required tools, e.g. `bin/cc`.

.bazelrc

@@ -1 +1,7 @@
 build --host_platform=@io_tweag_rules_nixpkgs//nixpkgs/platforms:host
+build --crosstool_top=@nixpkgs_config_cc//:toolchain
+# Using toolchain resolution can lead to spurious dependencies on
+# `@local_config_cc//:builtin_include_directory_paths`. This needs to be
+# resolved before `--incompatible_enable_cc_toolchain_resolution` can be
+# recommended for `nixpkgs_cc_configure_hermetic`.
+# build --incompatible_enable_cc_toolchain_resolution

CHANGELOG.md

@@ -8,6 +8,11 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/).
 
 [Unreleased]: https://github.com/tweag/rules_nixpkgs/compare/v0.7.0...HEAD
 
+### Added
+
+- Define `nixpkgs_cc_configure_hermetic` in `//nixpkgs:repositories.bzl`.
+  See [#128][#128].
+
 ### Changed
 
 - The values in the `nixopts` attribute to `nixpkgs_package` are now subject to
@@ -16,6 +21,13 @@ The format is based on [Keep a Changelog](https://keepachangelog.com/).
   `LABEL`. To pass a plain `$` to Nix it must be escaped as `$$`.
   See [#132][#132].
 
+### Deprecated
+
+- `nixpkgs_cc_configure` has been deprecated in favor of
+  `nixpkgs_cc_configure_hermetic` and will be replaced by it in future.
+  See [#128][#128].
+
+[#128]: https://github.com/tweag/rules_nixpkgs/pull/128
 [#132]: https://github.com/tweag/rules_nixpkgs/pull/132
 
 ## [0.7.0] - 2020-04-20

README.md

@@ -326,13 +326,116 @@ filegroup(
   </tbody>
 </table>
 
+### nixpkgs_cc_configure_hermetic
+
+Use a CC toolchain from Nixpkgs. No-op if not a nix-based platform.
+
+By default, Bazel auto-configures a CC toolchain from commands (e.g.
+`gcc`) available in the environment. To make builds more hermetic, use
+this rule to specify explicitly which commands the toolchain should use.
+
+Specifically, it builds a Nix derivation that provides the CC toolchain tools
+in the `bin/` path and constructs a CC toolchain that uses those tools. The
+following tools are expected `ar`, `cpp`, `dwp`, `cc`, `gcov`, `ld`, `nm`,
+`objcopy`, `objdump`, `strip`. Tools that aren't found are replaced by
+`${coreutils}/bin/false`.
+
+Note:
+
+You need to configure `--crosstool_top=@<name>//:toolchain` to activate this
+toolchain.
+
+Example:
+
+```bzl
+nixpkgs_cc_configure_hermetic(repository = "@nixpkgs//:default.nix")
+```
+
+<table class="table table-condensed table-bordered table-params">
+  <colgroup>
+    <col class="col-param" />
+    <col class="param-description" />
+  </colgroup>
+  <thead>
+    <tr>
+      <th colspan="2">Attributes</th>
+    </tr>
+  </thead>
+  <tbody>
+    <tr>
+      <td><code>attribute_path</code></td>
+      <td>
+        <p><code>String; optional</code></p>
+        <p>Obtain the toolchain from the Nix expression under this attribute path. Requires `nix_file` or `nix_file_content`.</p>
+      </td>
+      <td><code>nix_file</code></td>
+      <td>
+        <p><code>String; optional</code></p>
+        <p>Obtain the toolchain from the Nix expression defined in this file. Specify only one of `nix_file` or `nix_file_content`.</p>
+      </td>
+    </tr>
+    <tr>
+      <td><code>nix_file_content</code></td>
+      <td>
+        <p><code>String; optional</code></p>
+        <p>Obtain the toolchain from the given Nix expression. Specify only one of `nix_file` or `nix_file_content`.</p>
+      </td>
+    </tr>
+    <tr>
+      <td><code>nix_file_deps</code></td>
+      <td>
+        <p><code>List of labels; optional</code></p>
+        <p>Additional files that the Nix expression depends on.</p>
+      </td>
+    </tr>
+    <tr>
+      <td><code>repository</code></td>
+      <td>
+        <p><code>Label; optional</code></p>
+        <p>Provides `<nixpkgs>`. Specify one of `repositories` or `repository`.</p>
+      </td>
+    </tr>
+    <tr>
+      <td><code>repositories</code></td>
+      <td>
+        <p><code>String-keyed label dict; optional</code></p>
+        <p>Provides `<nixpkgs>` and other repositories. Specify one of `repositories` or `repository`.</p>
+      </td>
+    </tr>
+    <tr>
+      <td><code>quiet</code></td>
+      <td>
+        <p><code>Bool; optional</code></p>
+        <p>Whether to hide `nix-build` output.</p>
+      </td>
+    </tr>
+    <tr>
+      <td><code>fail_not_supported</code></td>
+      <td>
+        <p><code>Bool; optional</code></p>
+        <p>Whether to fail if `nix-build` is not available.</p>
+      </td>
+    </tr>
+  </tbody>
+</table>
+
 ### nixpkgs_cc_configure
 
 Tells Bazel to use compilers and linkers from Nixpkgs for the CC
 toolchain. By default, Bazel autodetects a toolchain on the current
 `PATH`. Overriding this autodetection makes builds more hermetic and
 is considered a best practice.
 
+Deprecated:
+
+Use `nixpkgs_cc_configure_hermetic` instead.
+
+While this improves upon Bazel's autoconfigure toolchain by picking tools from
+a Nix derivation rather than the environment, it is still not fully hermetic as
+it is affected by the environment. In particular, system include directories
+specified in the environment can leak in and affect the cache keys of targets
+depending on the cc toolchain leading to cache misses.
+
 Example:
 
 ```bzl

WORKSPACE

@@ -6,7 +6,7 @@ rules_nixpkgs_dependencies()
 
 load(
     "//nixpkgs:nixpkgs.bzl",
-    "nixpkgs_cc_configure",
+    "nixpkgs_cc_configure_hermetic",
     "nixpkgs_git_repository",
     "nixpkgs_local_repository",
     "nixpkgs_package",
@@ -161,7 +161,7 @@ nixpkgs_package(
     repository = "@nixpkgs",
 )
 
-nixpkgs_cc_configure(repository = "@remote_nixpkgs")
+nixpkgs_cc_configure_hermetic(repository = "@remote_nixpkgs")
 
 nixpkgs_python_configure(
     python2_attribute_path = "python2",