Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

adds layers to function #1835

Merged
merged 21 commits into from
Oct 9, 2024
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 6 additions & 0 deletions .changeset/calm-buttons-sort.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,6 @@
---
'@aws-amplify/backend-function': minor
'@aws-amplify/backend': minor
---

adds referenceLayer to function
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this and some code comments don't seem to be accurate now.

64 changes: 45 additions & 19 deletions packages/backend-function/src/factory.ts
Original file line number Diff line number Diff line change
@@ -1,3 +1,9 @@
import {
FunctionOutput,
functionOutputKey,
} from '@aws-amplify/backend-output-schemas';
import { AttributionMetadataStorage } from '@aws-amplify/backend-output-storage';
import { AmplifyUserError, TagName } from '@aws-amplify/platform-core';
import {
BackendOutputStorageStrategy,
BackendSecret,
Expand All @@ -12,28 +18,23 @@ import {
ResourceProvider,
SsmEnvironmentEntry,
} from '@aws-amplify/plugin-types';
import { Construct } from 'constructs';
import { NodejsFunction, OutputFormat } from 'aws-cdk-lib/aws-lambda-nodejs';
import * as path from 'path';
import { getCallerDirectory } from './get_caller_directory.js';
import { Duration, Stack, Tags } from 'aws-cdk-lib';
import { CfnFunction, Runtime } from 'aws-cdk-lib/aws-lambda';
import { createRequire } from 'module';
import { FunctionEnvironmentTranslator } from './function_env_translator.js';
import { Rule } from 'aws-cdk-lib/aws-events';
import * as targets from 'aws-cdk-lib/aws-events-targets';
import { Policy } from 'aws-cdk-lib/aws-iam';
import { CfnFunction, ILayerVersion, Runtime } from 'aws-cdk-lib/aws-lambda';
import { NodejsFunction, OutputFormat } from 'aws-cdk-lib/aws-lambda-nodejs';
import { Construct } from 'constructs';
import { readFileSync } from 'fs';
import { createRequire } from 'module';
import { fileURLToPath } from 'node:url';
import { EOL } from 'os';
import {
FunctionOutput,
functionOutputKey,
} from '@aws-amplify/backend-output-schemas';
import * as path from 'path';
import { FunctionEnvironmentTranslator } from './function_env_translator.js';
import { FunctionEnvironmentTypeGenerator } from './function_env_type_generator.js';
import { AttributionMetadataStorage } from '@aws-amplify/backend-output-storage';
import { fileURLToPath } from 'node:url';
import { AmplifyUserError, TagName } from '@aws-amplify/platform-core';
import { getCallerDirectory } from './get_caller_directory.js';
import { LayerReference, resolveLayers } from './reference_layer.js';
import { convertFunctionSchedulesToRuleSchedules } from './schedule_parser.js';
import * as targets from 'aws-cdk-lib/aws-events-targets';
import { Rule } from 'aws-cdk-lib/aws-events';

const functionStackType = 'function-Lambda';

Expand All @@ -52,6 +53,7 @@ export type TimeInterval =
| `every month`
| `every year`;
export type FunctionSchedule = TimeInterval | CronSchedule;
export type FunctionLayerReferences = Record<string, LayerReference>;

/**
* Entry point for defining a function in the Amplify ecosystem
Expand Down Expand Up @@ -121,6 +123,20 @@ export type FunctionProps = {
* schedule: "0 9 * * 2" // every Monday at 9am
*/
schedule?: FunctionSchedule | FunctionSchedule[];

/**
* Attach Lambda layers to a function
ykethan marked this conversation as resolved.
Show resolved Hide resolved
* @example
* import { referenceFunctionLayer } from "@aws-amplify/backend";
*
* layers: {
* myModule: referenceFunctionLayer("arn:aws:lambda:<current-region>:<account-id>:layer:myLayer:1")
* },
ykethan marked this conversation as resolved.
Show resolved Hide resolved
*
* The object is keyed by the module name hosted on your existing layer and a value that references to an existing layer using an ARN. The keys will be externalized and available via your layer at runtime
* Maximum of 5 layers can be attached to a function. Layers must be in the same region as the function.
ykethan marked this conversation as resolved.
Show resolved Hide resolved
*/
layers?: FunctionLayerReferences;
Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do we need referenceFunctionLayer ?

Can't we just

layers: {
   myModule: "arn:aws:lambda:<current-region>:<account-id>:layer:myLayer:1"
},

and have construct/factory validate and resolve necessary pieces inside ?

Copy link
Member Author

@ykethan ykethan Aug 8, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

this was built according to the RFC requirements
#1549

};

/**
Expand Down Expand Up @@ -166,6 +182,7 @@ class FunctionFactory implements ConstructFactory<AmplifyFunction> {
environment: this.props.environment ?? {},
runtime: this.resolveRuntime(),
schedule: this.resolveSchedule(),
layers: this.props.layers ?? {},
};
};

Expand Down Expand Up @@ -265,7 +282,9 @@ class FunctionFactory implements ConstructFactory<AmplifyFunction> {
};
}

type HydratedFunctionProps = Required<FunctionProps>;
type HydratedFunctionProps = Required<FunctionProps> & {
layers: FunctionLayerReferences;
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

why do we need this? Wouldn't layers be automatically be present and required in HydratedFunctionProps

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

removed this


class FunctionGenerator implements ConstructContainerEntryGenerator {
readonly resourceGroupName = 'function';
Expand All @@ -279,10 +298,15 @@ class FunctionGenerator implements ConstructContainerEntryGenerator {
scope,
backendSecretResolver,
}: GenerateContainerEntryProps) => {
const resolvedLayers = resolveLayers(
this.props.layers,
scope,
this.props.name
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

should this be part of hydrateDefaults?

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

the layers requires the scope for the fromLayerVersionArn to convert to a ILayer.

return new AmplifyFunction(
scope,
this.props.name,
this.props,
{ ...this.props, resolvedLayers },
backendSecretResolver,
this.outputStorageStrategy
);
Expand All @@ -301,7 +325,7 @@ class AmplifyFunction
constructor(
scope: Construct,
id: string,
props: HydratedFunctionProps,
props: HydratedFunctionProps & { resolvedLayers: ILayerVersion[] },
backendSecretResolver: BackendSecretResolver,
outputStorageStrategy: BackendOutputStorageStrategy<FunctionOutput>
) {
Expand Down Expand Up @@ -349,6 +373,7 @@ class AmplifyFunction
timeout: Duration.seconds(props.timeoutSeconds),
memorySize: props.memoryMB,
runtime: nodeVersionMap[props.runtime],
layers: props.resolvedLayers,
bundling: {
format: OutputFormat.ESM,
banner: bannerCode,
Expand All @@ -359,6 +384,7 @@ class AmplifyFunction
},
minify: true,
sourceMap: true,
externalModules: Object.keys(props.layers),
},
});
} catch (error) {
Expand Down
1 change: 1 addition & 0 deletions packages/backend-function/src/index.ts
Original file line number Diff line number Diff line change
@@ -1 +1,2 @@
export * from './factory.js';
export { referenceFunctionLayer } from './reference_layer.js';
144 changes: 144 additions & 0 deletions packages/backend-function/src/reference_layer.test.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,144 @@
import { StackMetadataBackendOutputStorageStrategy } from '@aws-amplify/backend-output-storage';
import {
ConstructContainerStub,
ResourceNameValidatorStub,
StackResolverStub,
} from '@aws-amplify/backend-platform-test-stubs';
import {
ConstructFactoryGetInstanceProps,
ResourceNameValidator,
} from '@aws-amplify/plugin-types';
import { App, Stack } from 'aws-cdk-lib';
import { Template } from 'aws-cdk-lib/assertions';
import assert from 'node:assert';
import { beforeEach, describe, it } from 'node:test';
import { defineFunction } from './factory.js';

const createStackAndSetContext = (): Stack => {
const app = new App();
app.node.setContext('amplify-backend-name', 'testEnvName');
app.node.setContext('amplify-backend-namespace', 'testBackendId');
app.node.setContext('amplify-backend-type', 'branch');
const stack = new Stack(app);
return stack;
};

void describe('AmplifyFunctionFactory - Layers', () => {
let rootStack: Stack;
let getInstanceProps: ConstructFactoryGetInstanceProps;
let resourceNameValidator: ResourceNameValidator;

beforeEach(() => {
rootStack = createStackAndSetContext();

const constructContainer = new ConstructContainerStub(
new StackResolverStub(rootStack)
);

const outputStorageStrategy = new StackMetadataBackendOutputStorageStrategy(
rootStack
);

resourceNameValidator = new ResourceNameValidatorStub();

getInstanceProps = {
constructContainer,
outputStorageStrategy,
resourceNameValidator,
};
});

void it('sets a valid layer', () => {
const layerArn = 'arn:aws:lambda:us-east-1:123456789012:layer:my-layer:1';
const functionFactory = defineFunction({
entry: './test-assets/default-lambda/handler.ts',
name: 'lambdaWithLayer',
layers: {
myLayer: {
arn: layerArn,
},
},
});
const lambda = functionFactory.getInstance(getInstanceProps);
const template = Template.fromStack(Stack.of(lambda.resources.lambda));

template.resourceCountIs('AWS::Lambda::Function', 1);
template.hasResourceProperties('AWS::Lambda::Function', {
Handler: 'index.handler',
Layers: [layerArn],
});
});

void it('sets multiple valid layers', () => {
const layerArns = [
'arn:aws:lambda:us-east-1:123456789012:layer:my-layer-1:1',
'arn:aws:lambda:us-east-1:123456789012:layer:my-layer-2:1',
];
const functionFactory = defineFunction({
entry: './test-assets/default-lambda/handler.ts',
name: 'lambdaWithMultipleLayers',
layers: {
myLayer1: {
arn: layerArns[0],
},
myLayer2: {
arn: layerArns[1],
},
},
});
const lambda = functionFactory.getInstance(getInstanceProps);
const template = Template.fromStack(Stack.of(lambda.resources.lambda));

template.resourceCountIs('AWS::Lambda::Function', 1);
template.hasResourceProperties('AWS::Lambda::Function', {
Handler: 'index.handler',
Layers: layerArns,
});
});

void it('throws an error for an invalid layer ARN', () => {
const invalidLayerArn = 'invalid:arn';
const functionFactory = defineFunction({
entry: './test-assets/default-lambda/handler.ts',
name: 'lambdaWithInvalidLayer',
layers: {
invalidLayer: {
arn: invalidLayerArn,
},
},
});
assert.throws(
() => functionFactory.getInstance(getInstanceProps),
new Error(`Invalid ARN format for layer invalidLayer: ${invalidLayerArn}`)
);
});

void it('throws an error for exceeding the maximum number of layers', () => {
const layerArns = [
'arn:aws:lambda:us-east-1:123456789012:layer:my-layer-1:1',
'arn:aws:lambda:us-east-1:123456789012:layer:my-layer-2:1',
'arn:aws:lambda:us-east-1:123456789012:layer:my-layer-3:1',
'arn:aws:lambda:us-east-1:123456789012:layer:my-layer-4:1',
'arn:aws:lambda:us-east-1:123456789012:layer:my-layer-5:1',
'arn:aws:lambda:us-east-1:123456789012:layer:my-layer-6:1',
];
const layers: Record<string, { arn: string }> = layerArns.reduce(
(acc, arn, index) => {
acc[`layer${index + 1}`] = { arn };
return acc;
},
{} as Record<string, { arn: string }>
);

const functionFactory = defineFunction({
entry: './test-assets/default-lambda/handler.ts',
name: 'lambdaWithTooManyLayers',
layers,
});

assert.throws(
() => functionFactory.getInstance(getInstanceProps),
new Error('A maximum of 5 unique layers can be attached to a function.')
);
});
});
77 changes: 77 additions & 0 deletions packages/backend-function/src/reference_layer.ts
Original file line number Diff line number Diff line change
@@ -0,0 +1,77 @@
import { ILayerVersion, LayerVersion } from 'aws-cdk-lib/aws-lambda';
import { Arn } from 'aws-cdk-lib/core';
import { Construct } from 'constructs';

/**
* Defines the ARN regex pattern for a layer.
*/
export const arnPattern = new RegExp(
'^(arn:(aws[a-zA-Z-]*)?:lambda:[a-z]{2}((-gov)|(-iso([a-z]?)))?-[a-z]+-\\d{1}:\\d{12}:layer:[a-zA-Z0-9-_]+:[0-9]+)|(arn:[a-zA-Z0-9-]+:lambda:::awslayer:[a-zA-Z0-9-_]+)$'

Check warning on line 9 in packages/backend-function/src/reference_layer.ts

View workflow job for this annotation

GitHub Actions / lint

You have a misspelled word: awslayer on String
);
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you add the source of this ARN in the documentation.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

got this regex from CFN error, but modified this now to be similar to the docs now and linked the documentation.


/**
* Checks if the provided ARN is valid.
*/
export const isValidLayerArn = (arn: string): boolean => {
return arnPattern.test(arn);
};
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

do we need these methods exported or used outside of this file? If not, can we bring them into the class as privates.

Copy link
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

separated the logic and added them as private.


/**
* Validates the provided layer ARNs and throws an error if any are invalid or if there are more than 5 unique ARNs.
*/
export const validateLayers = (
layers: Record<string, LayerReference>
): Set<string> => {
// Remove duplicate layer Arn's
const uniqueArns = new Set(Object.values(layers).map((layer) => layer.arn));

// Only 5 layers can be attached to a function
if (uniqueArns.size > 5) {
throw new Error(
'A maximum of 5 unique layers can be attached to a function.'
);
}

// Check if all Arn inputs are a valid Layer A
for (const [layerName, layerObj] of Object.entries(layers)) {
if (!isValidLayerArn(layerObj.arn)) {
throw new Error(
`Invalid ARN format for layer ${layerName}: ${layerObj.arn}`
);
}
}

return uniqueArns;
};

/**
* Type definition for a layer reference.
*/
export type LayerReference = { arn: string };

/**
* Creates a reference to a layer from an ARN.
*/
export const referenceFunctionLayer = (arn: string): LayerReference => {
return { arn };
};

/**
* Resolves and returns the layers for an AWS Lambda function.
*/
export const resolveLayers = (
layers: Record<string, LayerReference>,
scope: Construct,
functionName: string
): ILayerVersion[] => {
validateLayers(layers);
const uniqueArns = validateLayers(layers);
return Array.from(uniqueArns).map((arn) => {
const layerName = Arn.extractResourceName(arn, 'layer');
return LayerVersion.fromLayerVersionArn(
scope,
`${functionName}-${layerName}-layer`,
arn
);
});
};
9 changes: 6 additions & 3 deletions packages/backend/src/index.ts
Original file line number Diff line number Diff line change
@@ -1,12 +1,12 @@
export { defineBackend } from './backend_factory.js';
export * from './backend.js';
export { defineBackend } from './backend_factory.js';
export * from './secret.js';

// re-export core functionality from category packages

// data
export { defineData } from '@aws-amplify/backend-data';
export { type ClientSchema, a } from '@aws-amplify/data-schema';
export { a, type ClientSchema } from '@aws-amplify/data-schema';

// auth
export { defineAuth } from '@aws-amplify/backend-auth';
Expand All @@ -15,4 +15,7 @@ export { defineAuth } from '@aws-amplify/backend-auth';
export { defineStorage } from '@aws-amplify/backend-storage';

// function
export { defineFunction } from '@aws-amplify/backend-function';
export {
defineFunction,
referenceFunctionLayer,
} from '@aws-amplify/backend-function';
Loading