Skip to content

Commit

Permalink
feat!: add the ability to deprecate a prebuilt provider (#370)
Browse files Browse the repository at this point in the history
While this is not technically a breaking change, I want to force this to
be released as v0.5.0 so that it's easier to roll back to v0.4.x in case
there are any issues with this in testing.

At its core, this change is relatively simple: it introduces an
`isDeprecated` flag on the config for providers built using this project
that defaults to falsy. All the new behavior is controlled by that flag,
so it _should_ mean that this is safe to merge, as no new functionality
will be triggered for any providers unless/until they are marked as
deprecated.

(There technically is also a new option called `deprecationDate` -- in
practice I don't intend to use that and will just rely on the build
date; however, in order to not have the test snapshots be regenerated
every day with a new date, I needed a way to force-override the `new
Date()` behavior.)

If the `isDeprecated` flag is set to `true`, then the following happens:
- The "should-release" logic is removed because we want to force one
final release
- The "provider upgrade" and "upgrade-main" workflows are removed to
avoid any accidental updates before the repo is archived
- The text in the README will have a disclaimer about the deprecated
status of this package added to it, and the sections about Versioning
and Contributing that are no longer relevant will be removed
- Thanks to #371, these README changes will show up in the Go repository
as well
- The Go package will be marked as deprecated -- see
[here](golang/go#40357) for how this is done
in Go
- The NPM package will be [marked as
deprecated](https://docs.npmjs.com/deprecating-and-undeprecating-packages-or-package-versions/)
after the next and final release

PyPi and Maven unfortunately don't support deprecating an entire
package. NuGet does, but there's no API/CLI for it; it has to be done
via the web UI.

While it's huge, looking at the test snapshot file might be the best way
to visualize the changes. The first example in that file is of a
provider that's been marked as deprecated -- in particular, scrolling
down to the README will demonstrate what that looks like for a
deprecated provider. The other snapshots demonstrate that there
shouldn't be any unintentional outcomes for providers that have not been
explicitly marked as deprecated.

Side note: while I was working on this I realized that
hashicorp/terraform-cdk#2575 is very important now because we link to
https://cdk.tf/imports a lot to explain how to manually generate
bindings, but some of the examples there incorrectly use the prebuilt
providers, so I have added that issue to our next release.

---------

Signed-off-by: team-tf-cdk <[email protected]>
Co-authored-by: team-tf-cdk <[email protected]>
  • Loading branch information
xiehan and team-tf-cdk authored Dec 15, 2023
1 parent 3880437 commit fbad220
Show file tree
Hide file tree
Showing 8 changed files with 2,436 additions and 69 deletions.
30 changes: 30 additions & 0 deletions API.md

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

21 changes: 19 additions & 2 deletions src/cdktf-config.ts
Original file line number Diff line number Diff line change
Expand Up @@ -20,14 +20,30 @@ interface CdktfConfigOptions {
constructsVersion: string;
packageInfo: PackageInfo;
githubNamespace: string;
/**
* Whether or not this prebuilt provider is deprecated.
* If true, no new versions will be published.
*/
isDeprecated: boolean;
/**
* An optional date when the project should be considered deprecated,
* to be used in the README text. If no date is provided, then the
* date of the build will be used by default.
*/
deprecationDate?: string;
jsiiVersion?: string;
typescriptVersion?: string;
}

export class CdktfConfig {
constructor(project: cdk.JsiiProject, options: CdktfConfigOptions) {
const { terraformProvider, providerName, jsiiVersion, typescriptVersion } =
options;
const {
terraformProvider,
providerName,
jsiiVersion,
typescriptVersion,
isDeprecated,
} = options;

const cdktfVersion = options.cdktfVersion;
const constructsVersion = options.constructsVersion;
Expand Down Expand Up @@ -134,6 +150,7 @@ export class CdktfConfig {

project.addFields({
cdktf: {
isDeprecated,
provider: {
name: fullyQualifiedProviderName,
version: actualProviderVersion,
Expand Down
91 changes: 91 additions & 0 deletions src/deprecate-packages.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,91 @@
/**
* Copyright (c) HashiCorp, Inc.
* SPDX-License-Identifier: MPL-2.0
*/

import { JsiiProject } from "projen/lib/cdk";
import { JobPermission, JobStep } from "projen/lib/github/workflows-model";
import { PackageInfo } from "./package-info";

export interface DeprecatePackagesOptions {
providerName: string;
packageInfo: PackageInfo;
isDeprecated: boolean;
}

export class DeprecatePackages {
constructor(project: JsiiProject, options: DeprecatePackagesOptions) {
const { providerName, packageInfo, isDeprecated } = options;

const deprecationMessageForNPM = [
`See https://cdk.tf/imports for details on how to continue to use the ${providerName} provider`,
`in your CDK for Terraform (CDKTF) projects by generating the bindings locally.`,
].join(" ");
// @see https://github.com/golang/go/issues/40357
const deprecationMessageForGo = [
`// Deprecated: HashiCorp is no longer publishing new versions of the prebuilt provider for ${providerName}.`,
`// Previously-published versions of this prebuilt provider will still continue to be available as installable Go modules,`,
`// but these will not be compatible with newer versions of CDK for Terraform and are not eligible for commercial support.`,
`// You can continue to use the ${providerName} provider in your CDK for Terraform projects with newer versions of CDKTF,`,
`// but you will need to generate the bindings locally. See https://cdk.tf/imports for details.`,
].join("\n");

const deprecationStep: JobStep = {
name: "Mark the Go module as deprecated",
run: `find '.repo/dist/go' -mindepth 2 -maxdepth 4 -type f -name 'go.mod' | xargs sed -i '1s|^|${deprecationMessageForGo} \n|'`,
continueOnError: true, // @TODO remove this once we confirm it to be working
};
if (isDeprecated) {
packageInfo.publishToGo?.prePublishSteps?.splice(-1, 0, deprecationStep);
}

const releaseWorkflow = project.github?.tryFindWorkflow("release");
if (!releaseWorkflow) {
throw new Error("Could not find release workflow, aborting");
}

releaseWorkflow.addJobs({
deprecate: {
name: "Deprecate the package in package managers if needed",
runsOn: ["ubuntu-latest"],
needs: ["release", "release_npm"],
steps: [
{
name: "Checkout",
uses: "actions/checkout@v4",
},
{
name: "Install",
run: "yarn install",
},
{
name: "Check deprecation status",
id: "check_status",
run: [
`IS_DEPRECATED=$(npm pkg get cdktf.isDeprecated | tr -d '"')`,
`echo "is_deprecated=$IS_DEPRECATED"`, // for easier debugging
`echo "is_deprecated=$IS_DEPRECATED" >> $GITHUB_OUTPUT`,
].join("\n"),
},
{
name: "Deprecate the package on NPM",
if: "steps.check_status.outputs.is_deprecated",
run: `npm deprecate ${packageInfo.npm.name} "${deprecationMessageForNPM}"`,
env: {
NPM_REGISTRY: project.package.npmRegistry,
NPM_TOKEN:
"${{ secrets." + `${project.package.npmTokenSecret} }}`,
},
},
// Unfortunately, PyPi has no mechanism to mark a package as deprecated: https://github.com/pypi/warehouse/issues/345
// Maven also never moved past the proposal stage: https://github.com/p4em/artifact-deprecation
// NuGet supports deprecation, but only via the web UI: https://learn.microsoft.com/en-us/nuget/nuget-org/deprecate-packages
// Go deprecation is handled in the "Publish to Go" step (see line 37)
],
permissions: {
contents: JobPermission.READ,
},
},
});
}
}
121 changes: 76 additions & 45 deletions src/index.ts
Original file line number Diff line number Diff line change
Expand Up @@ -11,6 +11,7 @@ import { Automerge } from "./automerge";
import { CdktfConfig } from "./cdktf-config";
import { CopyrightHeaders } from "./copyright-headers";
import { CustomizedLicense } from "./customized-license";
import { DeprecatePackages } from "./deprecate-packages";
import { ForceRelease } from "./force-release";
import { GithubIssues } from "./github-issues";
import { LockIssues } from "./lock-issues";
Expand Down Expand Up @@ -51,6 +52,16 @@ export interface CdktfProviderProjectOptions extends cdk.JsiiProjectOptions {
* Will fall back to the current year if not specified.
*/
readonly creationYear?: number;
/**
* Whether or not this prebuilt provider is deprecated.
* If true, no new versions will be published.
*/
readonly isDeprecated?: boolean;
/**
* An optional date when the project should be considered deprecated, to be used in the README text.
* If no date is provided, then the date of the build will be used by default.
*/
readonly deprecationDate?: string;
}

const authorAddress = "https://hashicorp.com";
Expand Down Expand Up @@ -90,13 +101,16 @@ export class CdktfProviderProject extends cdk.JsiiProject {
minNodeVersion,
jsiiVersion,
typescriptVersion,
isDeprecated,
deprecationDate,
authorName = "HashiCorp",
namespace = "cdktf",
githubNamespace = "cdktf",
mavenEndpoint = "https://hashicorp.oss.sonatype.org",
nugetOrg = "HashiCorp",
mavenOrg = "hashicorp",
} = options;

const [fqproviderName, providerVersion] = terraformProvider.split("@");
const providerName = fqproviderName.split("/").pop();
assert(providerName, `${terraformProvider} doesn't seem to be valid`);
Expand Down Expand Up @@ -183,7 +197,8 @@ export class CdktfProviderProject extends cdk.JsiiProject {
// @see https://stackoverflow.com/a/49511949
"sed -i -e '/## Available Packages/,/### Go/!b' -e '/### Go/!d;p; s/### Go/## Go Package/' -e 'd' .repo/dist/go/*/README.md",
// sed -e is black magic and for whatever reason the string replace doesn't work so let's try it again:
"sed -i 's/### Go/## Go Package/' .repo/dist/go/*/README.md",
// eslint-disable-next-line prettier/prettier
`sed -i 's/### Go/## ${isDeprecated ? 'Deprecated' : 'Go'} Package/' .repo/dist/go/*/README.md`,
// Just straight up delete these full lines and everything in between them:
"sed -i -e '/API.typescript.md/,/You can also visit a hosted version/!b' -e 'd' .repo/dist/go/*/README.md",
`sed -i 's|Find auto-generated docs for this provider here:|Find auto-generated docs for this provider [here](https://${repositoryUrl}/blob/main/docs/API.go.md).|' .repo/dist/go/*/README.md`,
Expand Down Expand Up @@ -225,7 +240,6 @@ export class CdktfProviderProject extends cdk.JsiiProject {
devDeps: [
"@actions/core@^1.1.0",
"dot-prop@^5.2.0",
"semver@^7.5.3", // used by src/scripts/check-for-upgrades.ts
...(options.devDeps ?? []),
],
name: packageInfo.npm.name,
Expand All @@ -238,6 +252,7 @@ export class CdktfProviderProject extends cdk.JsiiProject {
repository: `https://github.com/${repository}.git`,
mergify: false,
eslint: false,
depsUpgrade: !isDeprecated,
depsUpgradeOptions: {
workflowOptions: {
labels: ["automerge", "auto-approve", "dependencies"],
Expand Down Expand Up @@ -308,9 +323,12 @@ export class CdktfProviderProject extends cdk.JsiiProject {
0,
setSafeDirectory
);
const { upgrade, pr } = (this.upgradeWorkflow as any).workflows[0].jobs;
upgrade.steps.splice(1, 0, setSafeDirectory);
pr.steps.splice(1, 0, setSafeDirectory);

if (!isDeprecated) {
const { upgrade, pr } = (this.upgradeWorkflow as any).workflows[0].jobs;
upgrade.steps.splice(1, 0, setSafeDirectory);
pr.steps.splice(1, 0, setSafeDirectory);
}

// Fix maven issue (https://github.com/cdklabs/publib/pull/777)
github.GitHub.of(this)?.tryFindWorkflow("release")?.file?.patch(
Expand All @@ -334,30 +352,35 @@ export class CdktfProviderProject extends cdk.JsiiProject {
typescriptVersion,
packageInfo,
githubNamespace,
});
const upgradeScript = new CheckForUpgradesScriptFile(this, {
providerVersion,
fqproviderName,
});
new ProviderUpgrade(this, {
checkForUpgradesScriptPath: upgradeScript.path,
workflowRunsOn,
nodeHeapSize: maxOldSpaceSize,
deprecationDate,
isDeprecated: !!isDeprecated,
});
new CustomizedLicense(this, options.creationYear);
new GithubIssues(this, { providerName });
new AutoApprove(this);
new AutoCloseCommunityIssues(this, { providerName });
new Automerge(this);
new LockIssues(this);
new AlertOpenPrs(this, {
slackWebhookUrl: "${{ secrets.ALERT_PRS_SLACK_WEBHOOK_URL }}",
repository,
});
new ForceRelease(this, {
workflowRunsOn,
repositoryUrl,
});

if (!isDeprecated) {
const upgradeScript = new CheckForUpgradesScriptFile(this, {
providerVersion,
fqproviderName,
});
new ProviderUpgrade(this, {
checkForUpgradesScriptPath: upgradeScript.path,
workflowRunsOn,
nodeHeapSize: maxOldSpaceSize,
});
new AlertOpenPrs(this, {
slackWebhookUrl: "${{ secrets.ALERT_PRS_SLACK_WEBHOOK_URL }}",
repository,
});
new ForceRelease(this, {
workflowRunsOn,
repositoryUrl,
});
}

new TextFile(this, ".github/CODEOWNERS", {
lines: [
Expand All @@ -369,31 +392,34 @@ export class CdktfProviderProject extends cdk.JsiiProject {
],
});

new ShouldReleaseScriptFile(this, {});
if (!isDeprecated) {
new ShouldReleaseScriptFile(this, {});

const releaseTask = this.tasks.tryFind("release")!;
this.removeTask("release");
this.addTask("release", {
description: releaseTask.description,
steps: releaseTask.steps,
env: (releaseTask as any)._env,
condition: "node ./scripts/should-release.js",
});
const releaseTask = this.tasks.tryFind("release")!;
this.removeTask("release");
this.addTask("release", {
description: releaseTask.description,
steps: releaseTask.steps,
env: (releaseTask as any)._env,
condition: "node ./scripts/should-release.js",
});

const releaseJobSteps: any[] = (
this.github?.tryFindWorkflow("release") as any
).jobs.release.steps;
const gitRemoteJob = releaseJobSteps.find((it) => it.id === "git_remote");
assert(
gitRemoteJob.run ===
'echo "latest_commit=$(git ls-remote origin -h ${{ github.ref }} | cut -f1)" >> $GITHUB_OUTPUT',
"git_remote step in release workflow did not match expected string, please check if the workaround still works!"
);
const previousCommand = gitRemoteJob.run;
const cancelCommand =
'echo "latest_commit=release_cancelled" >> $GITHUB_OUTPUT'; // this cancels the release via a non-matching SHA;
gitRemoteJob.run = `node ./scripts/should-release.js && ${previousCommand} || ${cancelCommand}`;
gitRemoteJob.name += " or cancel via faking a SHA if release was cancelled";
const releaseJobSteps: any[] = (
this.github?.tryFindWorkflow("release") as any
).jobs.release.steps;
const gitRemoteJob = releaseJobSteps.find((it) => it.id === "git_remote");
assert(
gitRemoteJob.run ===
'echo "latest_commit=$(git ls-remote origin -h ${{ github.ref }} | cut -f1)" >> $GITHUB_OUTPUT',
"git_remote step in release workflow did not match expected string, please check if the workaround still works!"
);
const previousCommand = gitRemoteJob.run;
const cancelCommand =
'echo "latest_commit=release_cancelled" >> $GITHUB_OUTPUT'; // this cancels the release via a non-matching SHA;
gitRemoteJob.run = `node ./scripts/should-release.js && ${previousCommand} || ${cancelCommand}`;
gitRemoteJob.name +=
" or cancel via faking a SHA if release was cancelled";
}

// Submodule documentation generation
this.gitignore.exclude("API.md"); // ignore the old file, we now generate it in the docs folder
Expand Down Expand Up @@ -454,6 +480,11 @@ export class CdktfProviderProject extends cdk.JsiiProject {
});

new CopyrightHeaders(this);
new DeprecatePackages(this, {
providerName,
packageInfo,
isDeprecated: !!isDeprecated,
});
}

private pinGithubActionVersions(pinnedVersions: Record<string, string>) {
Expand Down
Loading

0 comments on commit fbad220

Please sign in to comment.