-
Notifications
You must be signed in to change notification settings - Fork 2.6k
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
Added msal-node client-credential regression test (#6276)
This PR introduces the beginnings of regression testing for msal-node confidential client flows; In this specific case, the client credential flow. [BenchmarkJS](https://benchmarkjs.com/) is used to measure performance of msal-node. Sample output of BenchmarkJS looks like: <img width="1438" alt="Screenshot 2023-07-31 at 1 29 10 PM" src="https://github.com/AzureAD/microsoft-authentication-library-for-js/assets/87724641/f9dc302e-84b5-4ca8-820b-106502ab8684"> If a regression of >10% is detected, github actions will leave a commit comment: <img width="1882" alt="Screenshot 2023-07-31 at 1 28 40 PM" src="https://github.com/AzureAD/microsoft-authentication-library-for-js/assets/87724641/9e0a6252-baab-4c53-abee-69068bed3ed3"> [Example of commit comment when a fake regression was introduced to this PR.](917e9c4#commitcomment-123212353) Every time a PR is merged into dev, the latest output from "regression-tests/msal-node/client-credential/output.txt" (if available, if the client-credential test was run on the PR) will be saved to the branch "gh-pages", and will be available to visually see at: [https://azuread.github.io/microsoft-authentication-library-for-js/dev/bench/](https://azuread.github.io/microsoft-authentication-library-for-js/dev/bench/).
- Loading branch information
1 parent
2bc6389
commit a82c117
Showing
6 changed files
with
280 additions
and
0 deletions.
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,46 @@ | ||
name: msal-node client-credential Regression Test | ||
on: | ||
# Allows manual triggering of workflow | ||
workflow_dispatch: | ||
|
||
pull_request: | ||
types: [opened, reopened, synchronize, ready_for_review] | ||
paths: | ||
- "regression-tests/msal-node/client-credential/*" | ||
- "lib/msal-node/**/*" | ||
- "lib/msal-common/**/*" | ||
- "!**.md" | ||
- ".github/workflows/client-credential.yml" | ||
|
||
merge_group: | ||
types: [checks_requested] | ||
|
||
jobs: | ||
benchmark: | ||
name: Run msal-node client-credential Regression Test | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- uses: actions/setup-node@v3 | ||
- name: Run benchmark | ||
run: cd regression-tests/msal-node/client-credential && npm install && node index.js | tee output.txt | ||
|
||
- name: Download previous benchmark data | ||
uses: actions/cache@v3 | ||
with: | ||
path: ./cache | ||
key: ${{ runner.os }}-benchmark | ||
|
||
- name: Store benchmark result | ||
uses: benchmark-action/github-action-benchmark@v1 | ||
with: | ||
name: msal-node client-credential Regression Test | ||
tool: 'benchmarkjs' | ||
output-file-path: regression-tests/msal-node/client-credential/output.txt | ||
external-data-json-path: ./cache/client-credential-benchmark-data.json | ||
github-token: ${{ secrets.GITHUB_TOKEN }} | ||
# Show alert with commit comment on detecting possible performance regression | ||
alert-threshold: '110%' | ||
comment-on-alert: true | ||
fail-on-alert: false | ||
alert-comment-cc-users: '@bgavrilMS @robbie-microsoft' |
35 changes: 35 additions & 0 deletions
35
.github/workflows/msal-node-confidential-client-benchmarks.yml
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,35 @@ | ||
# Do not run this workflow on pull request since this workflow has permission to modify contents. | ||
on: | ||
push: | ||
branches: | ||
- dev | ||
|
||
permissions: | ||
# deployments permission to deploy GitHub pages website | ||
deployments: write | ||
# contents permission to update benchmark contents in gh-pages branch | ||
contents: write | ||
|
||
jobs: | ||
benchmark: | ||
name: Performance regression check | ||
runs-on: ubuntu-latest | ||
steps: | ||
- uses: actions/checkout@v3 | ||
- uses: actions/setup-node@v3 | ||
- name: Run benchmark | ||
run: cd regression-tests/msal-node/client-credential && npm install && node index.js | tee output.txt | ||
|
||
# gh-pages branch is updated and pushed automatically with extracted benchmark data | ||
# By default, this action assumes that gh-pages is the GitHub Pages branch and that | ||
# /dev/bench is a path to put the benchmark dashboard page. They can be tweaked by | ||
# the gh-pages-branch, gh-repository and benchmark-data-dir-path inputs. | ||
- name: Store benchmark result | ||
uses: benchmark-action/github-action-benchmark@v1 | ||
with: | ||
name: msal-node client-credential Regression Test | ||
tool: 'benchmarkjs' | ||
output-file-path: regression-tests/msal-node/client-credential/output.txt | ||
github-token: ${{ secrets.GITHUB_TOKEN }} | ||
# Push and deploy GitHub pages branch automatically | ||
auto-push: true |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,94 @@ | ||
/* | ||
* Copyright (c) Microsoft Corporation. All rights reserved. | ||
* Licensed under the MIT License. | ||
*/ | ||
|
||
const NetworkUtils = class NetworkUtils { | ||
// eslint-disable-next-line @typescript-eslint/explicit-module-boundary-types | ||
static getNetworkResponse(headers, body, statusCode) { | ||
return { | ||
headers: headers, | ||
body: body, | ||
status: statusCode, | ||
}; | ||
} | ||
}; | ||
|
||
const DEFAULT_OPENID_CONFIG_RESPONSE = { | ||
headers: {}, | ||
status: 200, | ||
body: { | ||
token_endpoint: | ||
"https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token", | ||
token_endpoint_auth_methods_supported: [ | ||
"client_secret_post", | ||
"private_key_jwt", | ||
"client_secret_basic", | ||
], | ||
jwks_uri: | ||
"https://login.microsoftonline.com/{tenant}/discovery/v2.0/keys", | ||
response_modes_supported: ["query", "fragment", "form_post"], | ||
subject_types_supported: ["pairwise"], | ||
id_token_signing_alg_values_supported: ["RS256"], | ||
response_types_supported: [ | ||
"code", | ||
"id_token", | ||
"code id_token", | ||
"id_token token", | ||
], | ||
scopes_supported: ["openid", "profile", "email", "offline_access"], | ||
issuer: "https://login.microsoftonline.com/{tenant}/v2.0", | ||
request_uri_parameter_supported: false, | ||
userinfo_endpoint: "https://graph.microsoft.com/oidc/userinfo", | ||
authorization_endpoint: | ||
"https://login.microsoftonline.com/{tenant}/oauth2/v2.0/authorize", | ||
http_logout_supported: true, | ||
frontchannel_logout_supported: true, | ||
end_session_endpoint: | ||
"https://login.microsoftonline.com/{tenant}/oauth2/v2.0/logout", | ||
claims_supported: [ | ||
"sub", | ||
"iss", | ||
"cloud_instance_name", | ||
"cloud_instance_host_name", | ||
"cloud_graph_host_name", | ||
"msgraph_host", | ||
"aud", | ||
"exp", | ||
"iat", | ||
"auth_time", | ||
"acr", | ||
"nonce", | ||
"preferred_username", | ||
"name", | ||
"tid", | ||
"ver", | ||
"at_hash", | ||
"c_hash", | ||
"email", | ||
], | ||
tenant_region_scope: null, | ||
cloud_instance_name: "microsoftonline.com", | ||
cloud_graph_host_name: "graph.windows.net", | ||
msgraph_host: "graph.microsoft.com", | ||
rbac_url: "https://pas.windows.net", | ||
}, | ||
}; | ||
|
||
const CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT = { | ||
headers: {}, | ||
status: 200, | ||
body: { | ||
token_type: "Bearer", | ||
expires_in: 3599, | ||
ext_expires_in: 3599, | ||
refresh_in: 3598 / 2, | ||
access_token: "thisIs.an.accessT0ken", | ||
}, | ||
}; | ||
|
||
module.exports = { | ||
CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT, | ||
DEFAULT_OPENID_CONFIG_RESPONSE, | ||
NetworkUtils, | ||
}; |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,2 @@ | ||
node_modules/ | ||
package-lock.json |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,87 @@ | ||
/* | ||
* Copyright (c) Microsoft Corporation. All rights reserved. | ||
* Licensed under the MIT License. | ||
*/ | ||
|
||
/* eslint-disable @typescript-eslint/no-unused-vars */ | ||
|
||
import benchmark from "benchmark"; | ||
import * as msal from "@azure/msal-node"; | ||
import { | ||
CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT, | ||
DEFAULT_OPENID_CONFIG_RESPONSE, | ||
NetworkUtils, | ||
} from "../Constants.js"; | ||
|
||
const clientConfig = { | ||
auth: { | ||
clientId: "client_id", | ||
authority: "https://login.microsoftonline.com/tenant_id", | ||
knownAuthorities: ["https://login.microsoftonline.com/tenant_id"], | ||
clientSecret: "client_secret", | ||
}, | ||
system: { | ||
networkClient: new class CustomHttpClient { | ||
sendGetRequestAsync(url, options, cancellationToken) { | ||
return new Promise((resolve, reject) => { | ||
const networkResponse = NetworkUtils.getNetworkResponse( | ||
DEFAULT_OPENID_CONFIG_RESPONSE.headers, | ||
DEFAULT_OPENID_CONFIG_RESPONSE.body, | ||
DEFAULT_OPENID_CONFIG_RESPONSE.status, | ||
); | ||
resolve(networkResponse); | ||
}); | ||
} | ||
sendPostRequestAsync(url, options) { | ||
return new Promise((resolve, _reject) => { | ||
const networkResponse = NetworkUtils.getNetworkResponse( | ||
CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT.headers, | ||
CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT.body, | ||
CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT.status, | ||
); | ||
resolve(networkResponse); | ||
}); | ||
} | ||
} | ||
}, | ||
}; | ||
|
||
const NUM_CACHE_ITEMS = 10; | ||
|
||
const confidentialClientApplication = new msal.ConfidentialClientApplication(clientConfig); | ||
const firstResourceRequest = { | ||
scopes: ["resource-1/.default"], | ||
}; | ||
const lastResourceRequest = { | ||
scopes: [`resource-${NUM_CACHE_ITEMS}/.default`], | ||
}; | ||
|
||
(async () => { | ||
// pre populate the cache | ||
for (let i = 1; i <= NUM_CACHE_ITEMS; i++) { | ||
const request = { | ||
scopes: [`resource-${i}/.default`], | ||
}; | ||
await confidentialClientApplication.acquireTokenByClientCredential(request); | ||
} | ||
|
||
const suite = new benchmark.Suite(); | ||
suite | ||
.add("ConfidentialClientApplication#acquireTokenByClientCredential-fromCache-resourceIsFirstItemInTheCache", { | ||
"fn": async () => { | ||
await confidentialClientApplication.acquireTokenByClientCredential(firstResourceRequest); | ||
}, | ||
"minSamples": 150, | ||
}) | ||
.add("ConfidentialClientApplication#acquireTokenByClientCredential-fromCache-resourceIsLastItemInTheCache", { | ||
"fn": async () => { | ||
await confidentialClientApplication.acquireTokenByClientCredential(lastResourceRequest); | ||
}, | ||
"minSamples": 150, | ||
}) | ||
.on("cycle", (event) => { | ||
// eslint-disable-next-line no-console | ||
console.log(String(event.target)); | ||
}) | ||
.run({ "async": true }); | ||
})(); |
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Original file line number | Diff line number | Diff line change |
---|---|---|
@@ -0,0 +1,16 @@ | ||
{ | ||
"name": "client-credential", | ||
"version": "1.0.0", | ||
"description": "Client Credential Regression Test", | ||
"main": "index.js", | ||
"type": "module", | ||
"scripts": { | ||
"test": "echo \"Error: no test specified\" && exit 1" | ||
}, | ||
"author": "", | ||
"license": "MIT", | ||
"dependencies": { | ||
"@azure/msal-node": "^2.0.0", | ||
"benchmark": "^2.1.4" | ||
} | ||
} |