Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

CAMS-461 separate function app #1056

Merged
Merged
Show file tree
Hide file tree
Changes from 48 commits
Commits
Show all changes
50 commits
Select commit Hold shift + click to select a range
9ec0b41
Move lib up to backend directory to isolate az functions
btposey Dec 5, 2024
05ed5c8
Fix backend start path
btposey Dec 5, 2024
6abe1b2
Change workflow paths to api
jamesobrooks Dec 6, 2024
1135cff
Fix remaining path issues
jamesobrooks Dec 6, 2024
73ebee8
Fix backend zip creation
jamesobrooks Dec 6, 2024
d23584f
separate durable function and workflow accomodations
amorrow-flexion Dec 6, 2024
cbb097d
edited workflow to run ci properly on newly split functions
amorrow-flexion Dec 9, 2024
0a9eb9e
corrected paths in pushd
amorrow-flexion Dec 9, 2024
d4b9822
removed npm
amorrow-flexion Dec 9, 2024
eb0765a
moved app insights and alert creation into one place
amorrow-flexion Dec 9, 2024
8c6b769
added node dependency to backend
amorrow-flexion Dec 9, 2024
61d2ecc
added use-base for ci
amorrow-flexion Dec 9, 2024
9700e79
created use-bash in package.json and corrected pack
amorrow-flexion Dec 9, 2024
0a99c0a
fixed artifact path on build
amorrow-flexion Dec 9, 2024
8f80082
addressed todo in bicep
amorrow-flexion Dec 9, 2024
407dcd8
fixed sql vnet rule fir migration functionapp
amorrow-flexion Dec 9, 2024
33115ca
fixed webapp name
amorrow-flexion Dec 9, 2024
dbda733
fixed dependency issue in network subnets
amorrow-flexion Dec 9, 2024
180e6b9
added stop operation to migration slot
amorrow-flexion Dec 9, 2024
73270b7
commented out prelim steps for now fixed appi dependencies
amorrow-flexion Dec 9, 2024
6231e1e
corrected pack commands and az-delete-branch-resources
amorrow-flexion Dec 9, 2024
75756c5
fixed pack commands
amorrow-flexion Dec 9, 2024
7359f21
correctly used dirs in zip command for pack
amorrow-flexion Dec 9, 2024
6210a16
another attempt at pack commands
amorrow-flexion Dec 9, 2024
a10905d
final attempt
amorrow-flexion Dec 9, 2024
a8a6ec4
Refactor building AZ function packages
btposey Dec 9, 2024
b93ce03
Fix running locally vs packaging
btposey Dec 10, 2024
307cdd4
Limit redundant builds when starting locally
btposey Dec 10, 2024
e6b42af
Sync .env and local.settings.json with functions
btposey Dec 10, 2024
e27a24a
Use OUT env var for naming package archive
btposey Dec 10, 2024
f545c1a
Fix temp archive path and file name for CI pipeline
btposey Dec 10, 2024
8505c2a
Fix paths to archive in Github workflow
btposey Dec 10, 2024
7264fb4
Fix typo in functions archive path
btposey Dec 10, 2024
2fa3214
Simplify pack.sh env var param for file name
btposey Dec 10, 2024
3da41ae
Consolidate build dir back into dist
btposey Dec 10, 2024
824218d
Merge pull request #1057 from US-Trustee-Program/CAMS-461-brian
btposey Dec 10, 2024
f08ad21
Run clean:all for the backend instead of clean
jamesobrooks Dec 10, 2024
080fd0e
Ignore package-lock.json files
jamesobrooks Dec 10, 2024
9a83643
Fix lint scripts
jamesobrooks Dec 10, 2024
39e05ac
Remove unused file
jamesobrooks Dec 10, 2024
2edede5
Clean up use of clean scripts
jamesobrooks Dec 10, 2024
7f5bb80
Remove unused scripts
jamesobrooks Dec 10, 2024
9a08c59
Rename functions directory to function-apps
jamesobrooks Dec 10, 2024
5a684f1
uncommmented unit tests and security scans
amorrow-flexion Dec 10, 2024
edac5b2
removed commented out bicep code
amorrow-flexion Dec 10, 2024
511ba77
[pre-commit.ci] auto fixes from pre-commit.com hooks
pre-commit-ci[bot] Dec 10, 2024
e6e6b4a
Fix package.json entry point paths
btposey Dec 10, 2024
4a1baee
Fix erroneous change
jamesobrooks Dec 10, 2024
359779a
Fix functions -> function-apps renaming errors
btposey Dec 10, 2024
488acb8
updated stop slot to include migration function
amorrow-flexion Dec 10, 2024
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
The table of contents is too big for display.
Diff view
Diff view
  •  
  •  
  •  
21 changes: 14 additions & 7 deletions .github/workflows/continuous-deployment.yml
Original file line number Diff line number Diff line change
Expand Up @@ -29,7 +29,9 @@ on:
- "false"
- "true"
enableBicepDeployment:
description: "enableBicepDeployment: Execute main.bicep deployment for application component Azure resources"
description:
"enableBicepDeployment: Execute main.bicep deployment for application component Azure
resources"
required: true
default: "false"
type: choice
Expand Down Expand Up @@ -88,7 +90,7 @@ jobs:
unit-test-backend:
uses: ./.github/workflows/reusable-unit-test.yml
with:
path: backend/functions
path: backend
node-version: ${{ vars.NODE_VERSION }}

unit-test-common:
Expand All @@ -109,12 +111,13 @@ jobs:
secrets: inherit # pragma: allowlist secret
with:
nodeVersion: ${{ vars.NODE_VERSION }}
apiName: ${{ needs.setup.outputs.apiName }}
camsServerHostname: ${{ needs.setup.outputs.apiName }}.azurewebsites${{ vars.AZ_HOSTNAME_SUFFIX }}
camsServerHostname: ${{ needs.setup.outputs.apiFunctionName }}.azurewebsites${{ vars.AZ_HOSTNAME_SUFFIX }}
camsServerPort: ${{ vars.CAMS_SERVER_PORT }}
camsServerProtocol: ${{ vars.CAMS_SERVER_PROTOCOL }}
camsBasePath: ${{ vars.CAMS_BASE_PATH }}
webappName: ${{ needs.setup.outputs.webappName }}
apiFunctionName: ${{ needs.setup.outputs.apiFunctionName }}
migrationFunctionName: ${{ needs.setup.outputs.migrationFunctionName }}
environment: ${{ needs.setup.outputs.ghaEnvironment }}
launchDarklyEnvironment: ${{ vars.CAMS_LAUNCH_DARKLY_ENV }}

Expand All @@ -131,13 +134,16 @@ jobs:
unit-test-backend,
unit-test-common,
]
if: ((github.ref == 'refs/heads/main') || (github.ref == 'refs/heads/dependency-updates-auto') || (inputs.deployBranch == 'true'))
if:
((github.ref == 'refs/heads/main') || (github.ref == 'refs/heads/dependency-updates-auto') ||
(inputs.deployBranch == 'true'))
with:
ghaEnvironment: ${{ needs.setup.outputs.ghaEnvironment }}
azResourceGrpAppEncrypted: ${{ needs.setup.outputs.azResourceGrpAppEncrypted }}
azResourceGrpNetworkEncrypted: ${{ needs.setup.outputs.azResourceGrpNetworkEncrypted }}
stackName: ${{ needs.setup.outputs.stackName }}
apiName: ${{ needs.setup.outputs.apiName }}
apiFunctionName: ${{ needs.setup.outputs.apiFunctionName }}
migrationFunctionName: ${{ needs.setup.outputs.migrationFunctionName }}
webAppName: ${{ needs.setup.outputs.webappName }}
environmentHash: ${{ needs.setup.outputs.environmentHash }}
execVnetDeploy: ${{ needs.setup.outputs.execVnetDeploy }}
Expand All @@ -154,7 +160,8 @@ jobs:
ghaEnvironment: ${{ needs.setup.outputs.ghaEnvironment }}
azResourceGrpAppEncrypted: ${{ needs.setup.outputs.azResourceGrpAppEncrypted }}
stackName: ${{ needs.setup.outputs.stackName }}
apiName: ${{ needs.setup.outputs.apiName }}
apiFunctionName: ${{ needs.setup.outputs.apiFunctionName }}
migrationFunctionName: ${{ needs.setup.outputs.migrationFunctionName }}
webAppName: ${{ needs.setup.outputs.webappName }}
environmentHash: ${{ needs.setup.outputs.environmentHash }}
slotName: ${{ needs.setup.outputs.slotName }}
Expand Down
14 changes: 10 additions & 4 deletions .github/workflows/reusable-backend-slot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ on:
webAppName:
required: true
type: string
apiName:
apiFunctionName:
required: true
type: string
migrationFunctionName:
required: true
type: string
ghaEnvironment:
Expand Down Expand Up @@ -51,16 +54,19 @@ jobs:

- name: Create API Slot Resource
run: |
slotStorageAccountName=ustpslot${{ inputs.environmentHash }}
migrationSlotStorageAccountName=ustpslot${{ inputs.environmentHash }}
apiSlotStorageAccountName=migrationslot${{ inputs.environmentHash }}

./ops/scripts/pipeline/slots/az-slot-api-resource-deploy.sh \
--resourceGroup ${{ steps.rgApp.outputs.out }} \
--idResourceGroup ${{ secrets.AZURE_RG }} \
--apiName ${{ inputs.apiName }} \
--apiFunctionName ${{ inputs.apiFunctionName }} \
--migrationFunctionName ${{ inputs.migrationFunctionName }} \
--webappName ${{ inputs.webappName }} \
--slotName ${{ inputs.slotName }} \
--kvIdName ${{ secrets.AZ_KV_APP_CONFIG_MANAGED_ID }} \
--sqlIdName ${{ secrets.AZ_SQL_IDENTITY_NAME }} \
--storageAccName ${slotStorageAccountName:0:24} \
--apiStorageAccName ${migrationSlotStorageAccountName:0:24} \
--migrationStorageAccName ${apiSlotStorageAccountName:0:24} \
--databaseName ${{ secrets.AZ_COSMOS_DATABASE_NAME }} \
--infoSha ${{ inputs.environmentHash || 'DOES_NOT_EXIST' }}
25 changes: 16 additions & 9 deletions .github/workflows/reusable-build-info.yml
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ on:
webappName:
description: "Expected Web Application name [used for Azure Resource]"
value: ${{ jobs.build-info.outputs.webappName }}
apiName:
apiFunctionName:
description: ""
value: ${{ jobs.build-info.outputs.apiName }}
value: ${{ jobs.build-info.outputs.apiFunctionName }}
azResourceGrpAppEncrypted:
description: ""
value: ${{ jobs.build-info.outputs.azResourceGrpAppEncrypted }}
Expand All @@ -31,6 +31,9 @@ on:
environmentHash:
description: ""
value: ${{ jobs.build-info.outputs.environmentHash }}
migrationFunctionName:
description: ""
value: ${{ jobs.build-info.outputs.migrationFunctionName }}
slotName:
description: ""
value: ${{ jobs.build-info.outputs.slotName }}
Expand All @@ -56,12 +59,13 @@ jobs:
outputs:
stackName: ${{ steps.build-info.outputs.stackName }}
webappName: ${{ steps.build-info.outputs.webappName }}
apiName: ${{ steps.build-info.outputs.apiName }}
apiFunctionName: ${{ steps.build-info.outputs.apiFunctionName }}
azResourceGrpAppEncrypted: ${{ steps.rgApp.outputs.out }}
azResourceGrpNetworkEncrypted: ${{ steps.rgNetwork.outputs.out }}
environmentHash: ${{ steps.check-env.outputs.environmentHash }}
execVnetDeploy: ${{ steps.build-info.outputs.execVnetDeploy }}
slotName: ${{ steps.build-info.outputs.slotName }}
migrationFunctionName: ${{ steps.build-info.outputs.migrationFunctionName }}
deployBicep: ${{ steps.check-bicep.outputs.deployBicep }}
initialDeployment: ${{ steps.check-initial-deployment.outputs.initialDeployment }}
steps:
Expand Down Expand Up @@ -101,9 +105,11 @@ jobs:
stackName=$(ops/scripts/pipeline/generate-stackname.sh ${{ inputs.environment }} ${{ vars.APP_NAME }} ${{ vars.DEV_SUFFIX }} ${{ github.ref_name }})
echo "Stack Name: $stackName"
webappName="${stackName}-webapp"
echo "Webapp Artifact: $webappName"
apiName="${stackName}-node-api"
echo "Api Artifact: $apiName"
echo "Webapp Name: $webappName"
apiFunctionName="${stackName}-node-api"
echo "Api Function Name: $apiFunctionName"
migrationFunctionName="${stackName}-migration"
echo "Migration Function Name: $migrationFunctionName"

# Deployment flags
echo "Bicep Deployment Flag: ${{ inputs.enableBicepDeployment == 'true' }}"
Expand All @@ -115,7 +121,8 @@ jobs:
echo "azResourceGrpNetwork=${{ secrets.AZ_NETWORK_RG }}${resourceGroupSuffix}" >> $GITHUB_OUTPUT
echo "stackName=${stackName}" >> $GITHUB_OUTPUT
echo "webappName=${webappName}" >> $GITHUB_OUTPUT
echo "apiName=${apiName}" >> $GITHUB_OUTPUT
echo "apiFunctionName=${apiFunctionName}" >> $GITHUB_OUTPUT
echo "migrationFunctionName=${migrationFunctionName}" >> $GITHUB_OUTPUT
echo "execVnetDeploy=${execVnetDeploy}" >> $GITHUB_OUTPUT
echo "slotName=${{ vars.SLOT_NAME }}" >> $GITHUB_OUTPUT

Expand All @@ -140,10 +147,10 @@ jobs:
op: decode
in: ${{ steps.rgApp.outputs.out }}

- name: Check Initial Deployment
- name: Check Initial Deployment # update flag --apiName to --apiFunctionName
btposey marked this conversation as resolved.
Show resolved Hide resolved
id: check-initial-deployment
run: |
initialDeployment=$(./ops/scripts/pipeline/check-for-environment.sh --resource-group ${{ steps.rgAppDecode.outputs.out }} --webappName ${{ steps.build-info.outputs.webappName }} --apiName ${{ steps.build-info.outputs.apiName }})
initialDeployment=$(./ops/scripts/pipeline/check-for-environment.sh --resource-group ${{ steps.rgAppDecode.outputs.out }} --webappName ${{ steps.build-info.outputs.webappName }} --apiName ${{ steps.build-info.outputs.apiFunctionName }})
echo "initialDeployment=${initialDeployment}" >> $GITHUB_OUTPUT
echo "Initial Deployment: ${initialDeployment}"

Expand Down
7 changes: 5 additions & 2 deletions .github/workflows/reusable-database-deploy.yml
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,10 @@ on:
azResourceGrpNetworkEncrypted:
required: true
type: string
apiName:
apiFunctionName:
required: true
type: string
migrationFunctionName:
required: true
type: string
environmentHash:
Expand Down Expand Up @@ -57,7 +60,7 @@ jobs:
- name: Deploy Cosmos MongoDB
id: azure-mongodb-deploy
run: |
allowedNetworks=$(az network vnet subnet list -g ${{ steps.rgNetwork.outputs.out }} --vnet-name ${{ vars.AZ_NETWORK_VNET_NAME }} --query "[?name=='snet-${{ inputs.apiName }}'].id" -o json)
allowedNetworks=$(az network vnet subnet list -g ${{ steps.rgNetwork.outputs.out }} --vnet-name ${{ vars.AZ_NETWORK_VNET_NAME }} --query "[?name=='snet-${{ inputs.apiFunctionName }}' || name=='snet-${{inputs.migrationFunctionName}}'].id" -o json)

./ops/scripts/pipeline/az-cosmos-deploy.sh \
-g ${{ secrets.AZURE_RG }} \
Expand Down
8 changes: 4 additions & 4 deletions .github/workflows/reusable-e2e.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: End-to-end Tests
on:
workflow_call:
inputs:
apiName:
apiFunctionName:
required: true
type: string
slotName:
Expand Down Expand Up @@ -75,13 +75,13 @@ jobs:
fi

echo "Seeding E2E CosmosDB..."
curl -v -d '{"txIdOverride": '0'}' -H "Content-Type: application/json" https://${{ inputs.apiName }}.azurewebsites${{ vars.AZ_HOSTNAME_SUFFIX }}/api/orders-sync?x-ms-routing-name=${{ inputs.slotName }}
curl -v -d '{"txIdOverride": '0'}' -H "Content-Type: application/json" https://${{ inputs.apiFunctionName }}.azurewebsites${{ vars.AZ_HOSTNAME_SUFFIX }}/api/orders-sync?x-ms-routing-name=${{ inputs.slotName }}

- name: Restart API and Install Node dependencies
working-directory: test/e2e
run: |
az functionapp restart --name ${{ inputs.apiName }} --resource-group ${{ steps.rgApp.outputs.out }} & az webapp restart --name ${{ inputs.webappName }} --resource-group ${{ steps.rgApp.outputs.out }}
az functionapp restart --name ${{ inputs.apiName }} --resource-group ${{ steps.rgApp.outputs.out }} --slot staging & az webapp restart --name ${{ inputs.webappName }} --resource-group ${{ steps.rgApp.outputs.out }} --slot staging
az functionapp restart --name ${{ inputs.apiFunctionName }} --resource-group ${{ steps.rgApp.outputs.out }} & az webapp restart --name ${{ inputs.webappName }} --resource-group ${{ steps.rgApp.outputs.out }}
az functionapp restart --name ${{ inputs.apiFunctionName }} --resource-group ${{ steps.rgApp.outputs.out }} --slot staging & az webapp restart --name ${{ inputs.webappName }} --resource-group ${{ steps.rgApp.outputs.out }} --slot staging
sleep 10
npm ci
npx playwright install chromium msedge
Expand Down
6 changes: 3 additions & 3 deletions .github/workflows/reusable-endpoint-test.yml
Original file line number Diff line number Diff line change
Expand Up @@ -3,7 +3,7 @@ name: Endpoint Tests
on:
workflow_call:
inputs:
apiName:
apiFunctionName:
required: true
type: string
slotName:
Expand Down Expand Up @@ -33,7 +33,7 @@ jobs:
env:
stackName: ${{ inputs.stackName }}
webappName: ${{ inputs.webappName }}
apiName: ${{ inputs.apiName }}
apiFunctionName: ${{ inputs.apiFunctionName }}
branchHashId: ${{ inputs.branchHashId }}
priority: 200
steps:
Expand Down Expand Up @@ -62,7 +62,7 @@ jobs:
- name: Health check App Resources
run: |
./ops/scripts/pipeline/endpoint-test.sh \
--apiName ${{ inputs.apiName }} \
--apiFunctionName ${{ inputs.apiFunctionName }} \
--webappName ${{ inputs.webappName }} \
--hostSuffix ${{ vars.AZ_HOSTNAME_SUFFIX }} \
--gitSha ${{ github.sha }} \
Expand Down
4 changes: 2 additions & 2 deletions .github/workflows/reusable-frontend-slot.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,7 @@ on:
webAppName:
required: true
type: string
apiName:
apiFunctionName:
required: true
type: string
ghaEnvironment:
Expand Down Expand Up @@ -51,7 +51,7 @@ jobs:
./ops/scripts/pipeline/slots/az-slot-web-resource-deploy.sh \
--resourceGroup ${{ steps.rgApp.outputs.out }} \
--webappName ${{ inputs.webappName }} \
--apiName ${{ inputs.apiName }} \
--apiFunctionName ${{ inputs.apiFunctionName }} \
--slotName ${{ inputs.slotName }} \
--subscription ${{ secrets.AZURE_SUBSCRIPTION }} \
--network-rg ${{ steps.rgNetwork.outputs.out }} \
Expand Down
39 changes: 28 additions & 11 deletions .github/workflows/sub-build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -6,7 +6,10 @@ on:
nodeVersion:
required: true
type: string
apiName:
apiFunctionName:
required: true
type: string
migrationFunctionName:
required: true
type: string
camsServerHostname:
Expand Down Expand Up @@ -84,28 +87,42 @@ jobs:
backend:
defaults:
run:
working-directory: backend/functions
working-directory: backend

runs-on: ubuntu-latest
environment: ${{ inputs.environment }}

steps:
- uses: actions/checkout@v3

- name: Execute Build
run: |
pushd ../../common
npm ci
popd
npm run use-bash

npm run build-common

npm ci
npm run build

- name: Package Application
run: OUT=${{ inputs.apiName }} npm run pack
npm run build:api

npm run build:migration

- name: Package API
run: OUT=${{ inputs.apiFunctionName }} npm run pack

- name: Package Migration
run: OUT=${{ inputs.migrationFunctionName }} npm run pack:migration

- name: Upload API Azure Functions Artifact
uses: actions/[email protected]
with:
name: ${{ inputs.apiFunctionName }}-build
path: backend/function-apps/api/${{ inputs.apiFunctionName }}.zip
if-no-files-found: error

- name: Upload Node Azure Functions Artifact
- name: Upload Migration Azure Functions Artifact
uses: actions/[email protected]
with:
name: ${{ inputs.apiName }}-build
path: backend/functions/${{ inputs.apiName }}.zip
name: ${{ inputs.migrationFunctionName }}-build
path: backend/function-apps/migration/${{ inputs.migrationFunctionName }}.zip
if-no-files-found: error
Loading
Loading