From e7d46cfab828768c85aa6313eb0486474788d4d8 Mon Sep 17 00:00:00 2001 From: Phil Pennock Date: Mon, 24 Jan 2022 17:39:40 -0500 Subject: [PATCH 1/2] Fix README & template to match parser Fix the examples in the template to match the form actually needed by the parser, and remove the invalid `Proposed` status. Document the flow for new ADRs starting in `Accepted` status, to reflect that as soon as they've been merged they are Accepted. --- .readme.templ | 2 ++ README.md | 2 ++ adr-template.md | 2 +- 3 files changed, 5 insertions(+), 1 deletion(-) diff --git a/.readme.templ b/.readme.templ index 1b637dec..dbcc8530 100644 --- a/.readme.templ +++ b/.readme.templ @@ -29,6 +29,8 @@ Please see the [template](adr-template.md). The template body is a guideline. Fe After editing / adding a ADR please run `go run main.go > README.md` to update the embedded index. This will also validate the header part of your ADR. +Note that new ADRs are written with an initial state of `Accepted`, not `Proposed`, on the basis that once merged to `main` it is immediately Accepted. + ## Related Repositories * Server [nats-server](https://github.com/nats-io/nats-server) diff --git a/README.md b/README.md index e0b0ff03..8b2e9d84 100644 --- a/README.md +++ b/README.md @@ -100,6 +100,8 @@ Please see the [template](adr-template.md). The template body is a guideline. Fe After editing / adding a ADR please run `go run main.go > README.md` to update the embedded index. This will also validate the header part of your ADR. +Note that new ADRs are written with an initial state of `Accepted`, not `Proposed`, on the basis that once merged to `main` it is immediately Accepted. + ## Related Repositories * Server [nats-server](https://github.com/nats-io/nats-server) diff --git a/adr-template.md b/adr-template.md index 45349aa9..efe8279d 100644 --- a/adr-template.md +++ b/adr-template.md @@ -4,7 +4,7 @@ |--------|-----| |Date |YYYY-MM-DD| |Author |@, @| -|Status |`Proposed`, `Approved` `Partially Implemented`, `Implemented`| +|Status |Approved, Partially Implemented, Implemented| |Tags |jetstream, client, server| ## Context and Problem Statement From d5dce7da3d9fe904309d12f490cce09538be5462 Mon Sep 17 00:00:00 2001 From: Phil Pennock Date: Mon, 24 Jan 2022 17:41:17 -0500 Subject: [PATCH 2/2] Propose ADR-22: Client File Locations This documents a combination of existing practice of some clients and an optimal New World Order to better support Windows, to try to get something well documented for all clients to adhere to. --- README.md | 10 ++- adr/ADR-21.md | 2 +- adr/ADR-22.md | 197 ++++++++++++++++++++++++++++++++++++++++++++++++++ 3 files changed, 207 insertions(+), 2 deletions(-) create mode 100644 adr/ADR-22.md diff --git a/README.md b/README.md index 8b2e9d84..5657daea 100644 --- a/README.md +++ b/README.md @@ -26,7 +26,15 @@ This repo is used to capture architectural and design decisions as a reference o |[ADR-18](adr/ADR-18.md)|client|URL support for all client options| |[ADR-19](adr/ADR-19.md)|jetstream, client, kv, objectstore|API prefixes for materialized JetStream views:| |[ADR-20](adr/ADR-20.md)|jetstream, client, objectstore|JetStream based Object Stores| -|[ADR-21](adr/ADR-21.md)|client|NATS Configuration Contexts| +|[ADR-21](adr/ADR-21.md)|client, config|NATS Configuration Contexts| +|[ADR-22](adr/ADR-22.md)|client, config|Client File Locations| + +## Config + +|Index|Tags|Description| +|-----|----|-----------| +|[ADR-21](adr/ADR-21.md)|client, config|NATS Configuration Contexts| +|[ADR-22](adr/ADR-22.md)|client, config|Client File Locations| ## Jetstream diff --git a/adr/ADR-21.md b/adr/ADR-21.md index d72e9dab..454117da 100644 --- a/adr/ADR-21.md +++ b/adr/ADR-21.md @@ -5,7 +5,7 @@ |Date |2021-12-14| |Author |@ripienaar| |Status |Partially Implemented| -|Tags |client| +|Tags |client, config| ## Background diff --git a/adr/ADR-22.md b/adr/ADR-22.md new file mode 100644 index 00000000..5c0feea8 --- /dev/null +++ b/adr/ADR-22.md @@ -0,0 +1,197 @@ +# Client File Locations + +|Metadata|Value| +|--------|-----| +|Date |2022-01-24| +|Author |@philpennock| +|Status |Approved| +|Tags |client, config| + + +## Context and Problem Statement + +The configuration files for NATS clients should be in a coherent predictable location. +Clients in various languages should be able to know where to read and write data, +and administrators should know what file-system access is needed and where +sensitive key data might be stored. + +This document both unifies the location and acts as a living registry of the +namespace within the NATS configuration directory. + + +## Version History + +|Date |Revision| +|----------|--------| +|2022-01-24|Initial proposal| + + +## [Context | References | Prior Work] + +* [ADR-21][] on Configuration Contexts moves us closer to the model + described herein, but ADR-21 will need updating for the non-XDG platforms. +* The `nsc` and `ngs` commands use other locations, scattering files around. +* A Unix-centric model is the [XDG Base Directory Specification][xdgbase]. + + +## Design + +We mandate a standard schema for where all NATS clients should lay out their +configuration files, and guidance for migration and handling. + +Our goals are: + + 1. Keep files of like type together, while adhering to OS norms. + 2. Never break existing user configuration. + 3. Provide a predictable working path across Unix-like systems, both macOS + and XDG-compliant Unix system. + 4. Register which files and sub-directories exist, to act as a central + repository of such information, and record who can safely rely upon such + details. + +Rather than reinvent the wheel, we propose to use the layout schema of +[Adrian-George Boston's Go implementation of XDG][adrg/xdg], which uses +specific other locations for other operating systems, all documented in +tables in the front README. + +The documentation is specific enough to allow implementations in languages +other than Go to use the same layouts. For Go, the `adrg/xdg` implementation +has two dependencies, both of which we already depend upon in multiple +`nats-io` repositories: `github.com/stretchr/testify` and `golang.org/x/sys`. +The license is MIT and the code is stable but still seeing maintenance +updates. It supports, “most flavors of Unix, Windows, macOS, and Plan 9”. + +All configuration should live inside a directory called `nats`. Users should +not need to learn which other tools might drop files elsewhere and should be +able to, minimally, discover all the configuration by files or symbolic links +within such directories. For configuration files, on Unix, this means that +all per-tool configuration will be within the `~/.config/nats/` directory, +while managed data might be within the `~/.local/share/nats/` directory. +Environment variables and choice of OS will change that. + +Our one deviation from the [adrg/xdg][] pattern is that on macOS, we +additionally require that `~/.config/nats` work, but in new installs it should +be a symbolic link to the `~/Library/Application Support/nats` directory. +This is a little extra work, but warranted by both complying with platform +expectations and being consistent since `.config/` does often exist on macOS +platforms where users have installed additional Unix heritage tools. + + +### Migration + +Much of the existing client configuration involves authentication data such as +NKeys or combined `.creds` files. These are more likely than most to have +been moved by the users to secure storage and replaced with symbolic links or +be on encrypted mounts. As such, automatically migrating such data is fraught +with peril. + +For the time being, we instead simply mandate that files be accessible by the +new paths; it is acceptable for client libraries to detect that the new +location does not exist on disk but the old location does exist, and put in a +symbolic link at the new location pointing to the old location. + +Ideally, the users will migrate their data to clean up the symbolic links. + +At a later date, we _might_ choose to automatically migrate files, as long as +a sweep of the old location confirms that all entries are files or +directories, free of symlinks, and all on the same file-system. + +In the below descriptions of old locations, a phrase "unless overridden by an +environment variable" would be needed in many places to strictly cover all +contingencies; for clarity in the prose, we omit that to just describe the +defaults. +Similarly, unless specifically addressing an operating system, we'll use the +XDG locations for examples. + +### The `nats` CLI + +The `nats` CLI uses the XDG location logic, in a simplified form, so the data +has until now been written to `~/.config/nats/` always. On Unix systems this +remains valid; on other systems supporting symbolic links, we should point to +that location. + +The `nats` CLI has proven the utility of the configuration model we're moving +to. + +The configuration used by the `nats` CLI are the configuration contexts, which +are generic for client tooling and not specific to that tool, so we consider +the `context/` directory and `context.txt` file to be top-level configuration +items. + +### The `nsc` CLI + +The default locations of files for `nsc` can be overridden with the +environment variables: `$NSC_HOME`, `$NKEYS_PATH`, and `$NSC_CWD_ONLY`. + +In addition, `nsc` has traditionally detected an account system Operator in +the current directory and then ignored all environment variables to use the +current directory as a root. +This behavior is considered an authentication administrator mode and should +not be used by most tools. + +Existing client tools should continue to honor `$NSC_HOME` and `$NKEYS_PATH` +to avoid breaking scripts, but new client tools should not use those +variables. + +These files all now are under the ownership of the `nsc` concept, so live in +the `nsc/` sub-directory of the NATS locations. + +The old file `~/.nsc/nsc.json` is configuration and so would live under +`$XDG_CONFIG_HOME` or equivalent. Thus on Unix, `~/.config/nats/nsc/nsc.json` +should be used. + +The credentials and nkeys files are considered data, and so will live under +`~/.local/share/nats/nsc/stores/` and `~/.local/share/nats/nsc/nkeys/`. + + +### Other environment variables not yet pulled in + +Some client tools support `$NATS_KEY` and `$NATS_CERT` to refer to client +X.509 keys and certificates. + +## Consequences + +At some point in the future, we should attempt to migrate existing files in +their old locations to the new standard locations. We should leave behind +symbolic links in "the other direction" to prevent breakage. + +We should never remove the symbolic links from the old locations, it's up to +the users/administrators to do so when they're happy to do so. + + +## Registry + +* Configuration + + The `context/` directory and `context.txt` file are used as per [ADR-21][] + for configuration contexts. + - Library implementations of contexts should be aware of this. + - Most applications should avoid assumptions. + - Contexts can contain sensitive data, where a server uses password-based + authentication. + - compatibility location: strict XDG adherence on all platforms. + + The `nsc/` and `nkeys/` directories are reserved for client credentials + and the account system, as described above. + - Most tools should not need to be specifically aware in code of the + location of these; where credentials are passed in, they are typically + part of the site configuration. + - New tools should use [ADR-21][] configuration contexts to locate + relevant data. + - The `nkeys/` directory and `nsc/creds/` directory both contain sensitive + client secrets and should be protected. When creating these + directories, or files within, permissions should be set to be tighter + than the "umask", permitting access only for the user. + + The `ngs/` directory is reserved for use by one of the corporate + maintainers of NATS, Synadia Communications. + + The `install-channel.txt` file is used by some installers for tracking + which channel of available releases the installers/updaters should use. + It should contain a single line of text containing a single word, the + channel name. + + + + +[ADR-21]: ADR-21.md +[adrg/xdg]: https://github.com/adrg/xdg + "Adrian-George Bostan's implementation of XDG in Go" +[xdgbase]: https://specifications.freedesktop.org/basedir-spec/basedir-spec-latest.html + "XDG Base Directory Specification"