Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: Multi bucket support #1742

Merged
merged 102 commits into from
Aug 7, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
102 commits
Select commit Hold shift + click to select a range
f48bb2d
feat: init multi-bucket support
0618 Jul 11, 2024
3c49e6f
refactor: BackendOutput type
0618 Jul 11, 2024
a3baa32
update package-lock
0618 Jul 11, 2024
b1f2c85
temp
0618 Jul 12, 2024
c20f71d
feat: update schema and client_config
0618 Jul 13, 2024
df34ec0
feat: increment schema and client_config versions to 1.1
0618 Jul 13, 2024
ed78564
rename to buckets and remove friendlyName
0618 Jul 13, 2024
8b365c1
chore: add changeset
0618 Jul 13, 2024
423f6ba
chore: update API.md
0618 Jul 14, 2024
2ab396b
fix: remove duplicated code
0618 Jul 15, 2024
87f2fd9
fix: construct test
0618 Jul 15, 2024
0bf1075
feat: validate isDefault
0618 Jul 16, 2024
9d28f1a
chore: remove as any
0618 Jul 16, 2024
4b1a471
test: fix client_config_contributor
0618 Jul 16, 2024
421fd59
Merge branch 'main' into multi-bucket-support
0618 Jul 16, 2024
9c27c2c
chore: update package-lock
0618 Jul 16, 2024
17cae19
fix: getOutput
0618 Jul 16, 2024
85130e3
refactor: backend_output_client
0618 Jul 17, 2024
8729889
refactor: DefaultBackendOutputClient
0618 Jul 17, 2024
0913ad5
Merge branch 'main' into multi-bucket-support
0618 Jul 17, 2024
62b2a36
Merge branch 'main' into multi-bucket-support
0618 Jul 17, 2024
e8548cf
revert changes on addBackendOutputEntry, use appendToBackendOutputList
0618 Jul 17, 2024
bc4d412
update JSdocs for construct
0618 Jul 17, 2024
f6a1c12
make storageOutput more specific
0618 Jul 18, 2024
cddcd97
add factoryCounter and hasDefault
0618 Jul 18, 2024
2c03a6d
use factoryCounter as postfix
0618 Jul 18, 2024
3c1c486
update changeset
0618 Jul 18, 2024
77df31d
Merge branch 'main' into multi-bucket-support
0618 Jul 18, 2024
9739169
fix: buckets output schema
0618 Jul 18, 2024
ea59529
feat: add name to buckets
0618 Jul 18, 2024
1b08c09
feat: throw no isdefault error before deploy
0618 Jul 18, 2024
107bcf9
chore: remove post-deploy error
0618 Jul 19, 2024
2f932ec
refactor: use factoryCounter for policy
0618 Jul 19, 2024
d67854e
refactor: pass buckets to appendToBackendOutputList
0618 Jul 19, 2024
4dc5553
Merge branch 'main' into multi-bucket-support
0618 Jul 19, 2024
bacfe86
remove StorageOutputPayloadToStore
0618 Jul 19, 2024
14ddd73
fix construct test
0618 Jul 19, 2024
cb21531
revert BackendOutput type
0618 Jul 19, 2024
b776969
chore: update changeset
0618 Jul 19, 2024
0ded8f7
fix: client_config_contributer_v1 and test
0618 Jul 19, 2024
c1b09b4
fix: pin api-extractor to 7.40.0
0618 Jul 19, 2024
8079d28
chore: update API.md
0618 Jul 19, 2024
ec113d0
refactor: use global factoryCounter for policy
0618 Jul 20, 2024
3380bd9
refactor: use addBackendOutputEntry for default
0618 Jul 22, 2024
92c6299
test: fix storage construct test
0618 Jul 22, 2024
88ea17d
this works
sobolk Jul 22, 2024
96bc326
move DeepPartial
0618 Jul 22, 2024
07194c3
remove StorageBucketsPayload
0618 Jul 22, 2024
50ebda5
refactor: use Aspects to validate storage
0618 Jul 23, 2024
6ced7be
fix: add buckets to metadata
0618 Jul 23, 2024
98567de
fix: factory unit test
0618 Jul 23, 2024
036862b
fix: metadata_output_storage_strategy unit test
0618 Jul 23, 2024
c8a36f7
fix: not export AmplifyStorage
0618 Jul 24, 2024
9843705
refactor: addOrUpdateMetadata
0618 Jul 24, 2024
35527d3
Merge branch 'main' into multi-bucket-support
0618 Jul 25, 2024
b797249
update package-lock
0618 Jul 25, 2024
7b326b6
update package-lock
0618 Jul 25, 2024
ac7e04b
Manual Merge branch 'update-package-lock' into multi-bucket-support
0618 Jul 25, 2024
5bf1cff
Update packages/backend-storage/src/factory.ts
0618 Jul 25, 2024
37f4e44
Update packages/backend-storage/src/factory.ts
0618 Jul 25, 2024
f208b87
refactor: remove redundant code in StorageClientConfigContributor
0618 Jul 25, 2024
440de9e
test: add unit tests for construct and factory
0618 Jul 26, 2024
8647309
Merge branch 'main' into multi-bucket-support
0618 Jul 26, 2024
bc33e54
fix: one bucket no default case
0618 Jul 26, 2024
e9f6859
fix: unit test
0618 Jul 27, 2024
9bc296b
Merge branch 'main' into multi-bucket-support
0618 Jul 30, 2024
4ce8db5
test log currentCodebaseOutputs and npmOutputs
0618 Jul 30, 2024
cba6cec
fix: convert bucketName, storageRegion into bucket_name, aws_region
0618 Jul 30, 2024
5429861
fix: client_config_contributor_v1.test
0618 Jul 30, 2024
3a2dd2b
refactor: use Aspect to replact factory static
0618 Jul 31, 2024
865b947
fix: buckets keys
0618 Jul 31, 2024
ca0d12d
fix: unit test
0618 Jul 31, 2024
cff7306
fix MultipleDefaultBucketError and add more unit tests
0618 Jul 31, 2024
db34c50
remove policyCount static
0618 Aug 1, 2024
98414b0
Update packages/backend-storage/src/factory.ts
0618 Aug 1, 2024
99b483f
refactor StorageValidator
0618 Aug 1, 2024
179a1bb
Refactor: storeOutput to factory
0618 Aug 2, 2024
16999af
Merge branch 'main' into multi-bucket-support
0618 Aug 2, 2024
084bd2c
remove comment
0618 Aug 2, 2024
9cde018
Update packages/backend-storage/src/factory.ts
0618 Aug 5, 2024
3473440
Update packages/backend-storage/src/factory.ts
0618 Aug 5, 2024
d9bb2c2
Update packages/backend-storage/src/factory.ts
0618 Aug 5, 2024
53db426
Update packages/backend-storage/src/construct.ts
0618 Aug 5, 2024
5792d10
Revert "test log currentCodebaseOutputs and npmOutputs"
0618 Aug 5, 2024
cf0d456
rename defaultStorageFound
0618 Aug 5, 2024
0e07a9f
Merge branch 'main' into multi-bucket-support
0618 Aug 5, 2024
436df66
remove firstStorage from getInstance
0618 Aug 5, 2024
3fffaba
rename buckets to storage, and fix unit tests
0618 Aug 5, 2024
cb7cb4f
use isStorageProcessed
0618 Aug 5, 2024
e042ab1
fix construct test
0618 Aug 5, 2024
87a35ca
fix StackMetadataBackendOutputStorageStrategy unit test
0618 Aug 5, 2024
336a517
fix a typo to make lint happy
0618 Aug 6, 2024
9ade76a
Update packages/backend-storage/src/factory.ts
0618 Aug 6, 2024
f0d993e
Update packages/client-config/src/client-config-contributor/client_co…
0618 Aug 6, 2024
f27e98b
fix type error in client_config_contributer_v1
0618 Aug 6, 2024
e329d26
refactor: use early return
0618 Aug 6, 2024
a66fe37
test: fix test name
0618 Aug 6, 2024
0940ed6
refactor: move Aspects to another file
0618 Aug 6, 2024
1a7f095
refactor: remove this.node
0618 Aug 6, 2024
13f20cf
Update packages/backend-storage/src/storage_outputs_aspect.ts
0618 Aug 6, 2024
a825efe
add isMatch to match outputs
0618 Aug 6, 2024
079fd73
add unit test for aspects
0618 Aug 7, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
11 changes: 11 additions & 0 deletions .changeset/sharp-penguins-impress.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,11 @@
---
'@aws-amplify/deployed-backend-client': minor
'@aws-amplify/backend-output-schemas': minor
'@aws-amplify/backend-output-storage': minor
'@aws-amplify/backend-storage': minor
'@aws-amplify/client-config': minor
'@aws-amplify/plugin-types': minor
'@aws-amplify/backend': minor
---

support adding more than one bucket
57 changes: 37 additions & 20 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

12 changes: 12 additions & 0 deletions packages/backend-output-schemas/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -275,24 +275,29 @@ export const unifiedBackendOutputSchema: z.ZodObject<{
payload: z.ZodObject<{
bucketName: z.ZodString;
storageRegion: z.ZodString;
buckets: z.ZodOptional<z.ZodString>;
}, "strip", z.ZodTypeAny, {
bucketName: string;
storageRegion: string;
buckets?: string | undefined;
}, {
bucketName: string;
storageRegion: string;
buckets?: string | undefined;
}>;
}, "strip", z.ZodTypeAny, {
version: "1";
payload: {
bucketName: string;
storageRegion: string;
buckets?: string | undefined;
};
}, {
version: "1";
payload: {
bucketName: string;
storageRegion: string;
buckets?: string | undefined;
};
}>]>>;
"AWS::Amplify::Custom": z.ZodOptional<z.ZodDiscriminatedUnion<"version", [z.ZodObject<{
Expand Down Expand Up @@ -391,6 +396,7 @@ export const unifiedBackendOutputSchema: z.ZodObject<{
payload: {
bucketName: string;
storageRegion: string;
buckets?: string | undefined;
};
} | undefined;
"AWS::Amplify::Function"?: {
Expand Down Expand Up @@ -455,6 +461,7 @@ export const unifiedBackendOutputSchema: z.ZodObject<{
payload: {
bucketName: string;
storageRegion: string;
buckets?: string | undefined;
};
} | undefined;
"AWS::Amplify::Function"?: {
Expand Down Expand Up @@ -684,24 +691,29 @@ export const versionedStorageOutputSchema: z.ZodDiscriminatedUnion<"version", [z
payload: z.ZodObject<{
bucketName: z.ZodString;
storageRegion: z.ZodString;
buckets: z.ZodOptional<z.ZodString>;
}, "strip", z.ZodTypeAny, {
bucketName: string;
storageRegion: string;
buckets?: string | undefined;
}, {
bucketName: string;
storageRegion: string;
buckets?: string | undefined;
}>;
}, "strip", z.ZodTypeAny, {
version: "1";
payload: {
bucketName: string;
storageRegion: string;
buckets?: string | undefined;
};
}, {
version: "1";
payload: {
bucketName: string;
storageRegion: string;
buckets?: string | undefined;
};
}>]>;

Expand Down
7 changes: 7 additions & 0 deletions packages/backend-output-schemas/src/storage/v1.ts
Original file line number Diff line number Diff line change
@@ -1,9 +1,16 @@
import { z } from 'zod';

const bucketSchema = z.object({
name: z.string(),
bucketName: z.string(),
storageRegion: z.string(),
Amplifiyer marked this conversation as resolved.
Show resolved Hide resolved
});

export const storageOutputSchema = z.object({
version: z.literal('1'),
payload: z.object({
bucketName: z.string(),
storageRegion: z.string(),
buckets: z.string(z.array(bucketSchema)).optional(), // JSON serialized array of bucketSchema
}),
});
3 changes: 2 additions & 1 deletion packages/backend-output-storage/API.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,7 @@

import { BackendOutputEntry } from '@aws-amplify/plugin-types';
import { BackendOutputStorageStrategy } from '@aws-amplify/plugin-types';
import { DeepPartial } from '@aws-amplify/plugin-types';
import * as _os from 'os';
import { PackageJsonReader } from '@aws-amplify/platform-core';
import { Stack } from 'aws-cdk-lib';
Expand Down Expand Up @@ -37,7 +38,7 @@ export type Platform = 'Mac' | 'Windows' | 'Linux' | 'Other';
export class StackMetadataBackendOutputStorageStrategy implements BackendOutputStorageStrategy<BackendOutputEntry> {
constructor(stack: Stack);
addBackendOutputEntry: (keyName: string, backendOutputEntry: BackendOutputEntry) => void;
appendToBackendOutputList: (keyName: string, backendOutputEntry: BackendOutputEntry) => void;
appendToBackendOutputList: (keyName: string, backendOutputEntry: DeepPartial<BackendOutputEntry>) => void;
}

// (No @packageDocumentation comment for this package)
Expand Down
Loading
Loading