nixos/netbird: harden and extend options

[nazarewk]

Description of changes

I have recently extensively tested and fixed all features of Netbird in my own implementation of multi-instance Netbird installations.

While doing so I discovered another multi-instance implementation got merged into nixpkgs #246055 which is slightly different, but still a solid base to upstream the rest of my changes:

• running as DynamicUser it's own user with minimal set of permissions
  • it was there before, but was lacking some of capabilities,
• made some configurations situational
• add more unmanaged interface configurations
• quality of life improvements:
  • configure log level for each interface
  • optionally turn off starting during boot
  • openFirewall by default
  • add shortcuts/wrappers for each created instance

I think it's a pretty good time to upstream, because I will be extensively using it at work: just launched my first Colmena-managed NixOS into GCE.

There are plans to support multi-account connections on the same daemon in Q2/2024 (see the slack message), but it's not known what shape it will take at all.

I decided to implement following significant changes:

• instances must specify a port they will be listening on as it doesn't make much sense to give an immediately conflicting default,
• aliased tunnels to clients, because a word tunnel does not exist in Netbird's nomenclature (unlike some other VPNs) and is pretty misleading. Also clients.* play nicely with my plan to implement a server in near future.
• skipped destructuring expressions (eg: {name, ...}: name -> client: client.name) because they make the code very hard to follow and update with increased number of options,

Things done

• Built on platform(s)
  • x86_64-linux
  • aarch64-linux
  • x86_64-darwin
  • aarch64-darwin
• For non-Linux: Is sandboxing enabled in nix.conf? (See Nix manual)
  • sandbox = relaxed
  • sandbox = true
• Tested, as applicable:
  • NixOS test(s) (look inside nixos/tests)
  • and/or package tests
  • or, for functions and "core" functionality, tests in lib/tests or pkgs/test
  • made sure NixOS tests are linked to the relevant packages
• Tested compilation of all packages that depend on this change using nix-shell -p nixpkgs-review --run "nixpkgs-review rev HEAD". Note: all changes have to be committed, also see nixpkgs-review usage
• Tested basic functionality of all binary files (usually in ./result/bin/)
• 24.05 Release Notes (or backporting 23.05 and 23.11 Release notes)
  • (Package updates) Added a release notes entry if the change is major or breaking
  • (Module updates) Added a release notes entry if the change is significant
  • (Module addition) Added a release notes entry if adding a new NixOS module
• Fits CONTRIBUTING.md.

---

Add a 👍 reaction to pull requests you find important.

[Tom-Hubrecht]

This seems overly complicated, as it is only used for defaults, you should use the name for this

[nazarewk]

It is also used for the wrapper, but yeah might make more sense to not drop -wt0 from the default config. The raw "unwrapped" package should work with it anyway.

[nazarewk]

it made more sense when there were 3 wrappers

[Tom-Hubrecht]

There is no netbird server in this PR

[nazarewk]

yeah, but it is very much relevant for the test, there is no confirmation the client actually works until we have a server to connect to.

[Tom-Hubrecht]

This is very weird, nixpkgs has a machinery already to wrap programs c.f. https://nixos.org/manual/nixpkgs/stable/#fun-wrapProgram

[nazarewk]

I'm pretty sure it would be significantly longer with wrapProgram, something like:

mkDerivation {
	pname = "xxx";
    installPhase = 
	    let
	  		exports = pipe config.environment [
	          (mapAttrsToList (name: default:
	            "--set-default '${name}' '${default}'"))
	          (concatStringsSep " ")
	        ];
		in
		"wrapProgram ${lib.getExe' cfg.package "netbird"} $out/bin/netbird${config.suffix} ${exports}";
}

actually makes more sense after prototyping it and seeing with own eyes

[Tom-Hubrecht]

You can do something like

    stdenv.mkDerivation {
      intallPhase = ''
        wrapProgram ${lib.getExe cfg.package} $out/bin/netbird${config.suffix} ${
          concatMapStringsSep " " ({ name, value }: "--set-default '${name}' '${default}'") (
            attrsToList config.environment
          )
        }
      '';

The issue I had was not because of the size of the code, but reinventing the wheel ^^

[nazarewk]

not so short anymore:

let
  makeWrapperArgs = flatten (mapAttrsToList
    (key: value: [ "--set-default" key value ])
    config.environment
  );
in
pkgs.stdenv.mkDerivation {
  name = "${cfg.package.name}-wrapper-${name}";
  meta.mainProgram = config.name;
  nativeBuildInputs = with pkgs; [ makeWrapper ];
  phases = [ "installPhase" ];
  installPhase = ''
    mkdir -p "$out/bin"
    makeWrapper ${lib.getExe cfg.package} "$out/bin/${config.name}" \
      ${escapeShellArgs makeWrapperArgs}
  '';
}

[Tom-Hubrecht]

When the networks have different domains, there is no interference, the only issue is when then share the same domain

[nazarewk]

I don't think the domain is configurable at all on the Netbird server side?

[Tom-Hubrecht]

It is when you self-host your server

[nazarewk]

FYI:

Hello[@nazarewk, we plan a custom DNS zone support coming next quarter. That would allow you to create custom domains for applications and peers

https://netbirdio.slack.com/archives/C02KHAE8VLZ/p1707820381270479?thread_ts=1707819591.188209&cid=C02KHAE8VLZ

[CertainLach]

Custom DNS zone support is already implemented for self-hosted,

        ${pkgs.netbird}/bin/netbird-mgmt management \
        --dns-domain dns.domain

Will place every machine under dns.domain subdomain: machine1.dns.domain, machine2.dns.domain

[PatrickDaG]

Module looks great. Hope it gets merged soon, been long enough.
I would like to test this more myself, but I'm working on a rewrite of the netbird-server modules right now and while I don't think they actually conflict, the merges do.

[PatrickDaG]

Should probably move this and the 2411 release notes to 2505.

[nazarewk]

to be honest, this PR is hanging for so many months I lost hope it will ever get merged and I stopped trying to chase people able to merge it.

[nazarewk]

I'll make the "proper" release notes when some of the maintainers able to give it review and merge pops up.
The code itself was written around 23.11 and already missed 2 releases.

[oddlama]

I know how frustrating a long running PR can be. The unfortunate thing is that any committer who comes along will see that this is missing proper release notes, and then can't merge it. So this will only delay it further.

I fully understand if you say that this requires too much work to keep up in the long term. Let me know if you need support with anything, otherwise I'd also offer pick this PR up if you don't want to keep working on it.

[PatrickDaG]

Do you plan to add more override files, or why do you first write the file in /etc and later iterate over the one file? I think it would be simpler to skip this and directly access the file from the nix-store in the derivation, skipping one indirection.

[nazarewk]

I'm using those on some of my machines and while testing stuff.

[PatrickDaG]

Should the client.ui.enable option then be removed as it doesn't do anything?

[PatrickDaG]

Probs would be better to use formatType.generate "config.json" cfg.config; since you're using the json format type already

[nazarewk]

don't think that's necessary, I think the generation uses jq with a separate derivation underneath.

[PatrickDaG]

There is an disable-auto-connect flag but it seems to only apply to netbird up, the link however seems to have run stale as there is no mention of such a feature being planned as far as I can see.

[nazarewk]

gotta look into it, I'm using some form of this in my configs successfully