Login.gov authentication - new packages infra-core, database, auth (#254

)

* login.gov sign-in working, roughly.

* Move login and logout to the header menu.

* For testing, trigger deploy on the login-gov branch

* Initial attempt at wiring login.gov credentials to deployed containers

* Add code to help manage secrets to the command-line interface.

* Create service-like command functions for secrets, and unit tests. Wire up in the command-line controller.

* Improve cli app organization

* Add set-login-gov-keys CLI command, use case, and tests.

* Add ADR for secrets management.

* Clarify README comment

* Fix bad comment

* Compare against correct value in test

* Add missing files

* Add key generation mocks in unit test.

* remove vitest reporter that hasn't been working for quite some time

* Set AUTH_SECRET for the auth library we're using

* Explicitly read the AUTH_SECRET in auto.config.ts, rather than rely on the framework to read.

* Move server secrets retrieval into secrets.ts

* experiment to revert - use hardcoded string for auth secret, in attempt to get tests passing in CI

* To improve readability, use literal strings in login.gov unit test

* Fix typing error in secrets package and organize its adapters into a separate subdirectory.

* Add "delete secret" command (so I can clean up mispelled keys)

* Test deployment to dev

* sanity check on reading an environment variable

* Used consistent AUTH_SECRET in CI tests; randomness is unnecessary

* Remove debug exception

* Refashion secrets packages as infra/core

* Remove unused docassemble cli commands

* Update infra references

* fix infra dir in validate workflow

* Fix infra-cdktf filtering in root package.json

* Log in flow wired up w/o sessions. Also, TODO: track the nonce.

* Add database package with an initial migration using Knex, with Kysely for query building.

* Add initial database package

* Organize database and auth code; trying to track down import problems when the Astro build is integrated at the app level. I'll factor out the auth code first.

* Create @atj/auth, and move logic in there.

* Initial auth flow working, but tests of auth services still TODO. Integration tests in the demo servers are still failing.

* Add tests for getProviderCallback and logOut.

* Add nonce

* Add createSession

* Test for processProviderCallback. Includes fix to reuse sqlite3 connection from knex.

* Add tests for process-session-cookie

* Manage database dependencies so the integrating form service apps may import and run the server's express handler. This includes:
- Create helper in @atj/common to create service objects from a collection of service functions
- Remove module-global imports of @atj/database from the @atj/server, replacing with async imports where appropriate
TODO: The app integration tests for Kansas and DOJ are now passing, but the build bundle fails.

* Instantiate SSMClient in the AWSParameterStoreSecretsVault constructor

* Update dependencies, fix issue with dev server mode not working, rename DatabaseService to DatabaseGateway

* Work to get the app integrations running again, which primarily involved using .js imports, so we are consistent with ESM loading rules.

* Trying to get the dependency graph command to work again. Will revisit later.

* Use git commit hash rather than deployment identifier as the deployment's docker tag

* Upgrade more dependencies (trying to get coverage working, and failing)

* Remove tsup usage from libraries

* Fix infra/cdktf filter in build script

* Use correct base tsconfig in the design package.

* Update design testing libraries

* Punt of getting code coverage working, for now.
Context: the design lib has an unhandled exception when coverage is enabled.

* Comment out davelosert/[email protected]

* In the Terraform PR plan comment, use the target branch of the PR rather than a hard-coded "staging" value

* Update e2e docker base image to newer playwright version

* Upgrade Storybook to 8.2. 8.0 caused some problems on one dev machine, so hoping the issues are resolved now.

* Fix missing .js extensions on db adapters

* Upgrade @storybook/test-runner to 0.19.1 (fixes build break)

* Wire client secret from app to login.gov provider in deployed apps.

* Remove passage of private key to auth - it shouldn't be necessary with PKCE. Revisit if we switch to the JWT method.

* Make clientSecret undefined rather than an empty string

* Remove clientSecret from auth test

* Remove clientSecret from LoginGovOptions

* Add extra debug logging for failed authorization code checks

* Remove deploy trigger from login-gov feature branch

* Remove unnecessary login.gov bash script (superceded by CLI commands)

* Remove old comment

* Add comment explaining `createService`

* Remove lucia from web server package, in lieu of the auth package.

.github/workflows/_terraform-apply.yml

@@ -53,7 +53,7 @@ jobs:
 
       - name: Initialize Terraform CDK configuration
         shell: bash
-        working-directory: infra
+        working-directory: infra/cdktf
         run: |
           pnpm cdktf get
           pnpm build:tsc
@@ -68,7 +68,7 @@ jobs:
           AWS_ACCESS_KEY_ID: ${{ secrets.AWS_ACCESS_KEY_ID }}
           AWS_SECRET_ACCESS_KEY: ${{ secrets.AWS_SECRET_ACCESS_KEY }}
         shell: bash
-        working-directory: infra
+        working-directory: infra/cdktf
         run: |
           cf api https://api.fr.cloud.gov
           DEPLOY_ENV=${DEPLOY_ENV} pnpm cdktf deploy --auto-approve

.github/workflows/_terraform-plan-pr-comment.yml

@@ -55,7 +55,7 @@ jobs:
 
       - name: Initialize Terraform CDK configuration
         shell: bash
-        working-directory: infra
+        working-directory: infra/cdktf
         run: |
           pnpm cdktf get
           pnpm build:tsc
@@ -70,19 +70,19 @@ jobs:
           cf api https://api.fr.cloud.gov
 
       - name: Synthesize Terraform configuration
-        working-directory: infra
+        working-directory: infra/cdktf
         run: |
           DEPLOY_ENV=${DEPLOY_ENV} pnpm cdktf synth
 
       - name: Get Terraform stack name
         id: get_stack_name
-        working-directory: infra
+        working-directory: infra/cdktf
         run: |
           DEPLOY_ENV=${DEPLOY_ENV} pnpm cdktf output --outputs-file outputs.json
           echo "stack_name=$(jq -r 'keys[0]' outputs.json)" >> $GITHUB_OUTPUT
 
       - name: Create Terraform plan
         uses: dflook/terraform-plan@v1
         with:
-          path: infra/cdktf.out/stacks/${{ steps.get_stack_name.outputs.stack_name }}
+          path: infra/cdktf/cdktf.out/stacks/${{ steps.get_stack_name.outputs.stack_name }}
           label: ${{ steps.get_stack_name.outputs.stack_name }}

.github/workflows/_validate.yml

@@ -56,19 +56,19 @@ jobs:
 
       - name: Run test suite
         shell: bash
-        run: pnpm test:ci
+        run: AUTH_SECRET=not-super-secret pnpm test:ci
 
       - name: Initialize Terraform CDK configuration
         shell: bash
         run: |
-          cd infra
+          cd infra/cdktf
           pnpm cdktf get
           pnpm build:tsc
 
       - name: Typecheck source code
         shell: bash
         run: pnpm typecheck
 
-      #- name: Vitest Coverage Report
-      #  if: always()
-      #  uses: davelosert/vitest-coverage-report-action@v2.2.0
+      # - name: Vitest Coverage Report
+      #   if: always()
+      #   uses: davelosert/vitest-coverage-report-action@v2.5.0

.github/workflows/add-terraform-plan-to-pr.yml

@@ -15,4 +15,4 @@ jobs:
     uses: ./.github/workflows/_terraform-plan-pr-comment.yml
     secrets: inherit
     with:
-      deploy-env: staging
+      deploy-env: ${{ github.base_ref }}

.github/workflows/deploy.yml

@@ -27,4 +27,4 @@ jobs:
     uses: ./.github/workflows/_terraform-apply.yml
     secrets: inherit
     with:
-      deploy-env: ${{ github.ref_name }}
+      deploy-env: ${{ github.ref_name }}

.nvmrc

@@ -1 +1 @@
-v20.15.0
+v20.16.0

apps/cli/README.md

@@ -1,13 +1,11 @@
 # @atj/cli-app
 
-This package includes a very simple command-line interface.
+This package defines the platform's command-line interface.
 
-## Example commands
-
-Examples:
+Commands are defined to aid with platform management operations. To see available commands, run:
 
 ```bash
-pnpm run cli create-workspace-graph
+pnpm cli --help
 ```
 
 ## Development

apps/cli/build.js

@@ -0,0 +1,14 @@
+const esbuild = require('esbuild');
+
+esbuild
+  .build({
+    bundle: true,
+    entryPoints: ['./src/index.ts'],
+    format: 'cjs',
+    minify: true,
+    outdir: './dist',
+    platform: 'node',
+    sourcemap: true,
+    target: 'es2020',
+  })
+  .catch(() => process.exit(1));

apps/cli/package.json

@@ -6,13 +6,14 @@
   "license": "CC0",
   "main": "src/index.ts",
   "scripts": {
-    "build": "echo 'skipping...' #tsc -p .",
-    "cli": "ts-node src/index.ts",
+    "build": "node ./build.js",
+    "cli": "node dist/index.js",
     "dev": "tsup src/* --watch",
     "test": "vitest run --coverage"
   },
   "dependencies": {
     "@atj/dependency-graph": "workspace:*",
+    "@atj/infra-core": "workspace:*",
     "commander": "^11.1.0"
   }
 }

apps/cli/src/cli-controller.test.ts → apps/cli/src/cli-controller/cli-controller.test.ts

@@ -1,18 +1,13 @@
 import { describe, expect, it, vi } from 'vitest';
 import { mock } from 'vitest-mock-extended';
 
-import { CliController } from './cli-controller';
+import { CliController } from '.';
 
 describe('cli controller', () => {
   it('works', async () => {
     const ctx = {
       console: mock<Console>({ log: vi.fn() }),
       workspaceRoot: '.',
-      docassemble: {
-        fetch: fetch,
-        apiUrl: '',
-        apiKey: '',
-      },
     };
     const app = CliController(ctx);
     await app.parseAsync(['node.js', 'dist/index.js', 'hello']);

apps/cli/src/cli-controller/index.ts

@@ -0,0 +1,30 @@
+import { Command } from 'commander';
+
+import { createDependencyGraph } from '@atj/dependency-graph';
+import type { Context } from './types';
+import { addSecretCommands } from './secrets';
+
+export const CliController = (ctx: Context) => {
+  const cli = new Command().description(
+    'CLI to interact with the ATJ workspace'
+  );
+
+  cli
+    .command('hello')
+    .description('say hello')
+    .action(() => {
+      ctx.console.log('Hello!');
+    });
+
+  cli
+    .command('create-workspace-graph')
+    .description('create a dependency graph of projects in the workspace')
+    .action(async () => {
+      await createDependencyGraph(ctx.workspaceRoot);
+      ctx.console.log('wrote workspace dependency graph');
+    });
+
+  addSecretCommands(ctx, cli);
+
+  return cli;
+};

apps/cli/src/cli-controller/secrets.ts

@@ -0,0 +1,85 @@
+import { Command } from 'commander';
+
+import { commands, getSecretsVault } from '@atj/infra-core';
+import { Context } from './types';
+import { type DeployEnv } from '@atj/infra-core/src/values';
+import path from 'path';
+
+export const addSecretCommands = (ctx: Context, cli: Command) => {
+  const cmd = cli
+    .command('secrets')
+    .option('-f, --file <path>', 'Source JSON file for secrets.', path => {
+      ctx.file = path;
+    })
+    .description('secrets management commands');
+
+  cmd
+    .command('delete')
+    .description('delete a secret')
+    .argument('<string>', 'secret key name')
+    .action(async (key: string) => {
+      const vault = await getSecretsVault(ctx.file);
+      await commands.deleteSecret(vault, key);
+    });
+
+  cmd
+    .command('get')
+    .description('get a secret value')
+    .argument('<string>', 'secret key name')
+    .action(async (key: string) => {
+      const vault = await getSecretsVault(ctx.file);
+      const secret = await commands.getSecret(vault, key);
+      console.log(secret);
+    });
+
+  cmd
+    .command('set')
+    .description('set a secret value')
+    .argument('<string>', 'secret key name')
+    .argument('<string>', 'secret value to set')
+    .action(async (key: string, value: string) => {
+      const vault = await getSecretsVault(ctx.file);
+      await commands.setSecret(vault, key, value);
+    });
+
+  cmd
+    .command('list')
+    .description('list all secret keys')
+    .action(async () => {
+      const vault = await getSecretsVault(ctx.file);
+      const secretKeys = await commands.getSecretKeyList(vault);
+      console.log(JSON.stringify(secretKeys, null, 2));
+    });
+
+  cmd
+    .command('show')
+    .description('show all secrets')
+    .action(async () => {
+      const vault = await getSecretsVault(ctx.file);
+      const allSecrets = await commands.getSecrets(vault);
+      console.log(JSON.stringify(allSecrets, null, 2));
+    });
+
+  cmd
+    .command('set-login-gov-keys')
+    .description(
+      'generate and save login.gov keypair; if it already exists, it is not ' +
+        'updated (future work might include adding key rotation)'
+    )
+    .argument('<deploy-env>', 'deployment environment (dev, staging)')
+    .argument('<app-key>', 'application key')
+    .action(async (env: DeployEnv, appKey: string) => {
+      const vault = await getSecretsVault(ctx.file);
+      const secretsDir = path.resolve(__dirname, '../../../infra/secrets');
+      const loginResult = await commands.setLoginGovSecrets(
+        { vault, secretsDir },
+        env,
+        appKey
+      );
+      if (loginResult.preexisting) {
+        console.log('Keypair already exists.');
+      } else {
+        console.log(`New keypair added`);
+      }
+    });
+};

apps/cli/src/cli-controller/types.ts

@@ -0,0 +1,5 @@
+export type Context = {
+  console: Console;
+  workspaceRoot: string;
+  file?: string;
+};

apps/cli/src/index.ts

@@ -1,6 +1,5 @@
-import { join } from 'path';
-import process from 'process';
-import { CliController } from './cli-controller';
+const { join } = require('path');
+const { CliController } = require('./cli-controller');
 
 // This should map to the directory containing the package.json.
 // By convention, assume that the originating process was run from the root
@@ -10,10 +9,5 @@ const workspaceRoot = join(process.cwd(), '../../');
 const app = CliController({
   console,
   workspaceRoot,
-  /*docassemble: {
-    fetch,
-    apiUrl: 'http://localhost:8011',
-    apiKey: process.env.VITE_DOCASSEMBLE_API_KEY || '',
-  },*/
 });
-app.parseAsync(process.argv).then(() => console.log('Done'));
+app.parseAsync(process.argv).then(() => console.error('Exiting...'));

apps/cli/tsconfig.json

@@ -1,7 +1,7 @@
 {
   "extends": "../../tsconfig.base.json",
   "compilerOptions": {
-    "module": "CommonJS",
+    "module": "ESNext",
     "outDir": "./dist",
     "emitDeclarationOnly": true
   },

apps/cli/tsup.config.ts

@@ -0,0 +1,11 @@
+import { defineConfig } from 'tsup';
+
+export default defineConfig({
+  entryPoints: ['src/index.ts'],
+  format: ['cjs', 'esm'],
+  target: 'es2020',
+  minify: true,
+  sourcemap: true,
+  clean: true,
+  bundle: true,
+});

apps/rest-api/package.json

@@ -13,7 +13,7 @@
     "@atj/forms": "workspace:*"
   },
   "devDependencies": {
-    "@types/aws-lambda": "^8.10.109",
+    "@types/aws-lambda": "^8.10.143",
     "esbuild": "^0.20.2"
   }
 }