Authenticate downloaded binaries

[wchargin]

Summary:
This pull request adds logic to authenticate all Bazel binaries that are
downloaded, as long as the user has GPG installed. If a user does not
have GPG installed, a new warning will be printed when a binary is
downloaded, but Bazelisk will function the same way as before. (GPG is
installed by default on Debian and Ubuntu.)

No new subprocesses are spawned when an already-downloaded version of
Bazel is run. The only appreciable overhead is incurred at download
time.

The Bazel public key and ownertrust information are included directly
into the bazelisk.py file so that the script can be self-contained; we
don’t have to worry about bundling it with any external assets.

Resolves #15.

Test Plan:
Download https://bazel.build/bazel-release.pub.gpg and verify that it
exactly matches the blob checked into this pull request:

$ curl --silent 'https://bazel.build/bazel-release.pub.gpg' | shasum -a 256
30af2ca7abfb65987cd61802ca6e352aadc6129dfb5bfc9c81f16617bc3a4416  -
$ sed -n '/BEGIN PGP/,/END PGP/p' bazelisk.py | shasum -a 256
30af2ca7abfb65987cd61802ca6e352aadc6129dfb5bfc9c81f16617bc3a4416  -

Then, follow these steps to check the behavior of the authentication:

• Remove the ~/.bazelisk directory. Run ./bazelisk.py version.
  Note that it downloads the latest binary and the latest signature,
  then prints “Authenticity verified” before invoking Bazel.

• Run ./bazelisk.py version again. Note that it does not verify the
  signature.

• Remove the ~/.bazelisk directory. Symlink /bin/false to
  ~/bin/gpg, and ensure that the symlink precedes the real gpg on
  your path. Run Bazelisk, and note that it prints a warning that GPG
  is not available but executes Bazel anyway. Run Bazelisk again, and
  note that it does not print the warning (because it reuses the
  existing executable without reauthenticating). Remove the symlink.

• Remove the ~/.bazelisk directory. Edit bazelisk.py, changing the
  determine_urls function so that the returned binary_url is an
  arbitrary web page (like http://example.com/) but the signature
  URL is unchanged. Run Bazelisk, and note that Bazelisk reports,
  “Failed to authenticate binary!”, includes the GPG output (“BAD
  signature”), and aborts with exit code 2 without invoking Bazel.
  Run ls ~/.bazelisk/bin and note that it does not include the
  invalid binary (though the signature is still there). Revert the
  changes to bazelisk.py.

• Remove the ~/.bazelisk directory. Create an arbitrary document and
  use gpg --detach-sign to sign it with a key that is not the Bazel
  signing key. Spawn a web server (python -m SimpleHTTPServer) to
  serve the “malicious executable” and its signature. Edit
  bazelisk.py, changing the determine_urls function to point both
  the binary and the signature to this local web server. Run Bazelisk,
  and note that it fails to authenticate the binary, with the message
  “public key not found”.

Repeat the above steps in Python 2 and Python 3.

Verify that your personal GnuPG database has not been modified (in
particular, the Bazel key should not have been installed, and the trust
settings should not have been modified).

I have tested this on Linux with gpg (GnuPG) 1.4.20. I don’t see any
reason that it shouldn’t work on macOS or Windows as long as the gpg(1)
interfaces are the same.

[philwo]

Code looks great, thanks a lot! I'll test it on my machines and will merge it then. :)

[philwo]

Hey! Sorry that it took me so long, but I finally added tests, a Buildkite CI config and they even pass on all three platforms. Would you mind rebasing and pushing, so that they run for your PR as well? :)

[wchargin]

No worries, and glad to hear it!

I’ve rebased and pushed, and the tests pass on Linux but fail on macOS
and Windows. I can’t find a way to view the output of the macOS
failures; I’m not even sure that the test target is being run. The only
error that I see is

[Errno 13] Permission denied: '/bazelisk_test'

even when I add a log statement to the very top of bazelisk_test.py.
But I see that the Buildkite tests pass on master, so it’s not clear to
me why we’d have trouble invoking the test target.

The Windows failure looks like it might be a bug in GPG: when invoked
with gpg --homedir D:\temp\foo, GPG is resolving its homedir to

Home: /d/temp/Bazel.runfiles_yq104_1d/runfiles/__main__/D:\temp\tmp_bazelisk_gpg_vmqe5mra

and then is confused when that directory does not exist. That’s somewhat
unfortunate, but I can probably work around it.

I’ll look into the Windows failure when I get a chance. Do you know what
might be going on with the macOS failure, or at least how I can see any
useful test output?

[philwo]

Thanks for the update!

I have no idea what that weird error on macOS is - it seems to come from Bazel after the test ran (or maybe from our CI script?). I'll look into it when I have some time.

I've modified the test and also added the flag --test_output=streamed on macOS, so you should see the output in the Buildkite log now, even if Bazel crashes afterwards. :)

[tmc]

this looks like it needs to be rebased

[pmuetschard]

Just a drive by comment. (I filed #97 and was referred to this)

[pmuetschard]

I think there should be a "project-owner" way of turning this into an error by default in order to use this on CI systems, for example, with confidence that the build is always built with an untainted bazel. I think using the .bazelversion file to specify a flag, something like --require-gpg-verification, could do the trick nicely.

[fenollp]

This isn't true for the Go version of bazelisk.