Skip to content

Commit

Permalink
Add @aws-amplify/backend-ai and @aws-amplify/constructs-ai with a…
Browse files Browse the repository at this point in the history
… conversation handler (#1948)

* Add `ai-constructs` and default implementation of conversation handler (#1822)

* add backend-ai package

* commonjs

* default handler - wip

* fix bedrock version

* fix that

* e2e test

* update response type

* update response type

* make code testable

* some PR feedback

* rename construct package

* top level try catch

* construct unit tests

* lint

* more tests

* ehhh

* adapter test

* more tests

* addressing TODOs

* readme

* undo that

* Add Bedrock tools to conversation handler (#1830)

* add tools

* changeset

* add e2e test for programmatic tool

* add e2e test for data tool

* lint

* adapter tests

* refactor event tools

* refactor event tools

* gql tool test

* provider test

* factory test

* detect all duplicates

* pr feedback.

* Merge main to conversation handler (#1846)

* update cdk error mapper to catch all deployment errors (#1826)

* update cdk error mapper to catch all deployment errors

* add package-loc

* prevent ctrl+c handling for pnpm (#1818)

* feat: Multi bucket support (#1742)

* feat: init multi-bucket support

* refactor: BackendOutput type

* update package-lock

* temp

* feat: update schema and client_config

* feat: increment schema and client_config versions to 1.1

* rename to buckets and remove friendlyName

* chore: add changeset

* chore: update API.md

* fix: remove duplicated code

* fix: construct test

* feat: validate isDefault

* chore: remove as any

* test: fix client_config_contributor

* chore: update package-lock

* fix: getOutput

* refactor: backend_output_client

* refactor: DefaultBackendOutputClient

* revert changes on addBackendOutputEntry, use appendToBackendOutputList

* update JSdocs for construct

* make storageOutput more specific

* add factoryCounter and hasDefault

* use factoryCounter as postfix

* update changeset

* fix: buckets output schema

* feat: add name to buckets

* feat: throw no isdefault error before deploy

* chore: remove post-deploy error

* refactor: use factoryCounter for policy

* refactor: pass buckets to appendToBackendOutputList

* remove StorageOutputPayloadToStore

* fix construct test

* revert BackendOutput type

* chore: update changeset

* fix: client_config_contributer_v1 and test

* fix: pin api-extractor to 7.40.0

* chore: update API.md

* refactor: use global factoryCounter for policy

* refactor: use addBackendOutputEntry for default

* test: fix storage construct test

* this works

* move DeepPartial

* remove StorageBucketsPayload

* refactor: use Aspects to validate storage

* fix: add buckets to metadata

* fix: factory unit test

* fix: metadata_output_storage_strategy unit test

* fix: not export AmplifyStorage

* refactor: addOrUpdateMetadata

* update package-lock

* update package-lock

* Update packages/backend-storage/src/factory.ts

Co-authored-by: Amplifiyer <[email protected]>

* Update packages/backend-storage/src/factory.ts

Co-authored-by: Amplifiyer <[email protected]>

* refactor: remove redundant code in StorageClientConfigContributor

* test: add unit tests for construct and factory

* fix: one bucket no default case

* fix: unit test

* test log currentCodebaseOutputs and npmOutputs

* fix: convert bucketName, storageRegion into bucket_name, aws_region

* fix: client_config_contributor_v1.test

* refactor: use Aspect to replact factory static

* fix: buckets keys

* fix: unit test

* fix MultipleDefaultBucketError and add more unit tests

* remove policyCount static

* Update packages/backend-storage/src/factory.ts

Co-authored-by: Amplifiyer <[email protected]>

* refactor StorageValidator

* Refactor: storeOutput to factory

* remove comment

* Update packages/backend-storage/src/factory.ts

Co-authored-by: Kamil Sobol <[email protected]>

* Update packages/backend-storage/src/factory.ts

Co-authored-by: Kamil Sobol <[email protected]>

* Update packages/backend-storage/src/factory.ts

Co-authored-by: Kamil Sobol <[email protected]>

* Update packages/backend-storage/src/construct.ts

Co-authored-by: Kamil Sobol <[email protected]>

* Revert "test log currentCodebaseOutputs and npmOutputs"

This reverts commit 4ce8db5.

* rename defaultStorageFound

* remove firstStorage from getInstance

* rename buckets to storage, and fix unit tests

* use isStorageProcessed

* fix construct test

* fix StackMetadataBackendOutputStorageStrategy unit test

* fix a typo to make lint happy

* Update packages/backend-storage/src/factory.ts

Co-authored-by: Amplifiyer <[email protected]>

* Update packages/client-config/src/client-config-contributor/client_config_contributor_v1.ts

Co-authored-by: Edward Foyle <[email protected]>

* fix type error in client_config_contributer_v1

* refactor: use early return

* test: fix test name

* refactor: move Aspects to another file

* refactor: remove this.node

* Update packages/backend-storage/src/storage_outputs_aspect.ts

Co-authored-by: Amplifiyer <[email protected]>

* add isMatch to match outputs

* add unit test for aspects

---------

Co-authored-by: Kamil Sobol <[email protected]>
Co-authored-by: Amplifiyer <[email protected]>
Co-authored-by: Kamil Sobol <[email protected]>
Co-authored-by: Edward Foyle <[email protected]>

* chore(deps): bump fast-xml-parser, @aws-sdk/client-amplify, @aws-sdk/client-cloudformation, @aws-sdk/client-cognito-identity-provider, @aws-sdk/client-dynamodb, @aws-sdk/client-iam, @aws-sdk/client-s3 and @aws-sdk/client-ssm (#1836)

Bumps [fast-xml-parser](https://github.com/NaturalIntelligence/fast-xml-parser) to 4.4.1 and updates ancestor dependencies [fast-xml-parser](https://github.com/NaturalIntelligence/fast-xml-parser), [@aws-sdk/client-amplify](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-amplify), [@aws-sdk/client-cloudformation](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-cloudformation), [@aws-sdk/client-cognito-identity-provider](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-cognito-identity-provider), [@aws-sdk/client-dynamodb](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-dynamodb), [@aws-sdk/client-iam](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-iam), [@aws-sdk/client-s3](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-s3) and [@aws-sdk/client-ssm](https://github.com/aws/aws-sdk-js-v3/tree/HEAD/clients/client-ssm). These dependencies need to be updated together.


Updates `fast-xml-parser` from 4.1.2 to 4.4.1
- [Release notes](https://github.com/NaturalIntelligence/fast-xml-parser/releases)
- [Changelog](https://github.com/NaturalIntelligence/fast-xml-parser/blob/master/CHANGELOG.md)
- [Commits](NaturalIntelligence/fast-xml-parser@v4.1.2...v4.4.1)

Updates `@aws-sdk/client-amplify` from 3.614.0 to 3.624.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-amplify/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.624.0/clients/client-amplify)

Updates `@aws-sdk/client-cloudformation` from 3.614.0 to 3.624.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-cloudformation/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.624.0/clients/client-cloudformation)

Updates `@aws-sdk/client-cognito-identity-provider` from 3.614.0 to 3.625.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-cognito-identity-provider/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.625.0/clients/client-cognito-identity-provider)

Updates `@aws-sdk/client-dynamodb` from 3.614.0 to 3.624.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-dynamodb/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.624.0/clients/client-dynamodb)

Updates `@aws-sdk/client-iam` from 3.614.0 to 3.624.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-iam/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.624.0/clients/client-iam)

Updates `@aws-sdk/client-s3` from 3.614.0 to 3.626.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-s3/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.626.0/clients/client-s3)

Updates `@aws-sdk/client-ssm` from 3.614.0 to 3.624.0
- [Release notes](https://github.com/aws/aws-sdk-js-v3/releases)
- [Changelog](https://github.com/aws/aws-sdk-js-v3/blob/main/clients/client-ssm/CHANGELOG.md)
- [Commits](https://github.com/aws/aws-sdk-js-v3/commits/v3.624.0/clients/client-ssm)

---
updated-dependencies:
- dependency-name: fast-xml-parser
  dependency-type: indirect
- dependency-name: "@aws-sdk/client-amplify"
  dependency-type: direct:production
- dependency-name: "@aws-sdk/client-cloudformation"
  dependency-type: direct:production
- dependency-name: "@aws-sdk/client-cognito-identity-provider"
  dependency-type: direct:development
- dependency-name: "@aws-sdk/client-dynamodb"
  dependency-type: direct:development
- dependency-name: "@aws-sdk/client-iam"
  dependency-type: direct:development
- dependency-name: "@aws-sdk/client-s3"
  dependency-type: direct:production
- dependency-name: "@aws-sdk/client-ssm"
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Revert "feat: Multi bucket support" (#1837)

* Revert "feat: Multi bucket support (#1742)"

This reverts commit d8b43d2.

* add changeset

* upgrade AWS SDK packages to latest (#1839)

* Split sandbox e2e tests (#1841)

* Handle nested namespaces in api check (#1842)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Amplifiyer <[email protected]>
Co-authored-by: Roshane Pascual <[email protected]>
Co-authored-by: MJ Zhang <[email protected]>
Co-authored-by: Edward Foyle <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>

* Make conversation response selection set dynamic (#1848)

* Make conversation selection set dynamic

* grrr

* UI tools (#1853)

* add client tools to schema

* api

* logic and unit tests

* e2e

* pr feedback

* Add `defineConversationHandlerFunction` and `@aws-amplify/backend-ai` package. (#1858)

* Add backend-ai

* factory

* this works

* changeset

* api

* fix that

* boop

* add unit test

* readme

* some pr feedback

* Serialize tool us blocks (#1866)

* stringify tool input

* stringify tool input

* Adjust API.md generation in backend-ai for api check to work. (#1869)

* Adjust API.md generation in backend-ai for api check to work.

* changeset

* Add main entry point. (#1870)

* fix e2e test

* Add ability to pass inference configuration (#1878)

* Add ability to pass inference configuration

* Add ability to pass inference configuration

* fix(ai-constructs): support no input graphql query tools (#1907)

* fix(ai-constructs): no input graphql query tools

* add changeset

* empty

* fix api diff

* fix tool query assertion

* lets try this again

* Api adjustments for conversation handler (#1938)

* accept models from schema def

* api updates

* api updates

* Squash changeset on feature/conversation-handler branch and fix api check (#1949)

* squash changeset

* fix api check

* grrr

* handle usage of subnamespaced types in api check (#1952)

---------

Signed-off-by: dependabot[bot] <[email protected]>
Co-authored-by: Amplifiyer <[email protected]>
Co-authored-by: Roshane Pascual <[email protected]>
Co-authored-by: MJ Zhang <[email protected]>
Co-authored-by: Edward Foyle <[email protected]>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
Co-authored-by: Ian Saultz <[email protected]>
  • Loading branch information
7 people authored Sep 4, 2024
1 parent 10ce672 commit 99c8b6a
Show file tree
Hide file tree
Showing 76 changed files with 3,917 additions and 56 deletions.
8 changes: 8 additions & 0 deletions .changeset/blue-tigers-cover.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,8 @@
---
'@aws-amplify/ai-constructs': minor
'@aws-amplify/backend-ai': minor
'@aws-amplify/backend-function': patch
'@aws-amplify/platform-core': minor
---

Add new @aws-amplify/ai-constructs and @aws-amplify/backend-ai packages with a conversation handler.
3 changes: 3 additions & 0 deletions .eslint_dictionary.json
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,9 @@
"amazoncognito",
"amplifyconfiguration",
"ampx",
"anthropic",
"apns",
"apollo",
"appleid",
"appsync",
"argv",
Expand All @@ -21,6 +23,7 @@
"changeset",
"changesets",
"chown",
"claude",
"cloudformation",
"codebase",
"codegen",
Expand Down
44 changes: 42 additions & 2 deletions package-lock.json

Some generated files are not rendered by default. Learn more about how customized files appear on GitHub.

14 changes: 14 additions & 0 deletions packages/ai-constructs/.npmignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,14 @@
# Be very careful editing this file. It is crafted to work around [this issue](https://github.com/npm/npm/issues/4479)

# First ignore everything
**/*

# Then add back in transpiled js and ts declaration files
!lib/**/*.js
!lib/**/*.d.ts

# Then ignore test js and ts declaration files
*.test.js
*.test.d.ts

# This leaves us with including only js and ts declaration files of functional code
125 changes: 125 additions & 0 deletions packages/ai-constructs/API.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,125 @@
## API Report File for "@aws-amplify/ai-constructs"

> Do not edit this file. It is a report generated by [API Extractor](https://api-extractor.com/).
```ts

/// <reference types="node" />

import * as bedrock from '@aws-sdk/client-bedrock-runtime';
import { Construct } from 'constructs';
import { DocumentType } from '@smithy/types';
import { FunctionResources } from '@aws-amplify/plugin-types';
import { ResourceProvider } from '@aws-amplify/plugin-types';

declare namespace conversation {
export {
ConversationHandlerFunction,
ConversationHandlerFunctionProps,
runtime
}
}
export { conversation }

// @public
class ConversationHandlerFunction extends Construct implements ResourceProvider<FunctionResources> {
constructor(scope: Construct, id: string, props: ConversationHandlerFunctionProps);
// (undocumented)
resources: FunctionResources;
}

// @public (undocumented)
type ConversationHandlerFunctionProps = {
entry?: string;
models: Array<{
modelId: string;
region?: string;
}>;
};

// @public (undocumented)
type ConversationMessage = {
role: 'user' | 'assistant';
content: Array<ConversationMessageContentBlock>;
};

// @public (undocumented)
type ConversationMessageContentBlock = bedrock.ContentBlock;

// @public (undocumented)
type ConversationTurnEvent = {
conversationId: string;
currentMessageId: string;
responseMutation: {
name: string;
inputTypeName: string;
selectionSet: string;
};
graphqlApiEndpoint: string;
modelConfiguration: {
modelId: string;
systemPrompt: string;
region?: string;
inferenceConfiguration?: {
maxTokens?: number;
temperature?: number;
topP?: number;
};
};
request: {
headers: {
authorization: string;
};
};
messages: Array<ConversationMessage>;
toolsConfiguration?: {
dataTools?: Array<ToolDefinition & {
graphqlRequestInputDescriptor: {
queryName: string;
selectionSet: string;
propertyTypes: Record<string, string>;
};
}>;
clientTools?: Array<ToolDefinition>;
};
};

// @public (undocumented)
type ExecutableTool = ToolDefinition & {
execute: (input: DocumentType | undefined) => Promise<ToolResultContentBlock>;
};

// @public
const handleConversationTurnEvent: (event: ConversationTurnEvent, props?: {
tools?: Array<ExecutableTool>;
}) => Promise<void>;

declare namespace runtime {
export {
ConversationMessage,
ConversationMessageContentBlock,
ConversationTurnEvent,
ExecutableTool,
handleConversationTurnEvent,
ToolDefinition,
ToolInputSchema,
ToolResultContentBlock
}
}

// @public (undocumented)
type ToolDefinition = {
name: string;
description: string;
inputSchema: ToolInputSchema;
};

// @public (undocumented)
type ToolInputSchema = bedrock.ToolInputSchema;

// @public (undocumented)
type ToolResultContentBlock = bedrock.ToolResultContentBlock;

// (No @packageDocumentation comment for this package)

```
66 changes: 66 additions & 0 deletions packages/ai-constructs/README.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,66 @@
# Description

This package vends L3 CDK Constructs that deploy resources which power AI routes in Amplify apps.

# Conversation Handler Function

Conversation Handler Function L3 CDK Construct provisions resources required to integrate conversation routes (i.e.
defined with `a.ai.conversation()` from `@aws-amplify/data-schema`) with AI models provided by AWS Bedrock service.

Resources deployed by this construct include:

1. AWS Lambda function that:
1. Accepts conversation turn events coming from upstream AWS AppSync instance that is hosting conversation routes
in a form of GraphQL schemas.
2. Interacts with AWS Bedrock to elicit a response to current conversation turn.
3. Sends back the response to the same AWS AppSync instance in a form of GraphQL mutation.
2. AWS CloudWatch Log Group associated with AWS Lambda function with addition of appropriate data protection policy.
3. AWS IAM Policy that grants access to selected AWS Bedrock models.

This construct is used implicitly by `@aws-amplify/data-construct` when conversational routes don't specify
handler reference. In this case default implementation provided by Amplify is deployed.

Alternatively, a custom lambda implementation can be provided by explicitly including this construct in backend definition
and referencing it in schema.

## Examples

### Default implementation

```typescript
import { App, Stack } from 'aws-cdk-lib';
import { ConversationHandlerFunction } from '@aws-amplify/ai-constructs/conversation';

const app = new App();
const stack = new Stack(app, 'ConversationHandlerStack');

new ConversationHandlerFunction(stack, {
models: [
{
modelId: 'anthropic.claude-3-haiku-20240307-v1:0',
region: 'us-east-1',
},
],
});
```

### Custom implementation

```typescript
import { App, Stack } from 'aws-cdk-lib';
import { ConversationHandlerFunction } from '@aws-amplify/ai-constructs/conversation';
import path from 'path';

const app = new App();
const stack = new Stack(app, 'ConversationHandlerStack');

new ConversationHandlerFunction(stack, {
entry: path.resolve('./custom_handler.ts'),
models: [
{
modelId: 'anthropic.claude-3-haiku-20240307-v1:0',
region: 'us-east-1',
},
],
});
```
3 changes: 3 additions & 0 deletions packages/ai-constructs/api-extractor.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,3 @@
{
"extends": "../../api-extractor.base.json"
}
36 changes: 36 additions & 0 deletions packages/ai-constructs/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,36 @@
{
"name": "@aws-amplify/ai-constructs",
"version": "0.0.1",
"type": "commonjs",
"publishConfig": {
"access": "public"
},
"exports": {
".": {
"types": "./lib/index.d.ts",
"default": "./lib/index.js"
},
"./conversation": {
"types": "./lib/conversation/index.d.ts",
"default": "./lib/conversation/index.js"
},
"./conversation/runtime": {
"types": "./lib/conversation/runtime/index.d.ts",
"default": "./lib/conversation/runtime/index.js"
}
},
"types": "lib/index.d.ts",
"scripts": {
"update:api": "api-extractor run --local"
},
"license": "Apache-2.0",
"dependencies": {
"@aws-amplify/plugin-types": "^1.0.1"
},
"peerDependencies": {
"@aws-sdk/client-bedrock-runtime": "^3.622.0",
"@smithy/types": "^3.3.0",
"aws-cdk-lib": "^2.152.0",
"constructs": "^10.0.0"
}
}
Loading

0 comments on commit 99c8b6a

Please sign in to comment.