Skip to content

Commit

Permalink
Added msal-node client-credential regression test (#6276)
Browse files Browse the repository at this point in the history
This PR introduces the beginnings of regression testing for msal-node
confidential client flows; In this specific case, the client credential
flow. [BenchmarkJS](https://benchmarkjs.com/) is used to measure
performance of msal-node.

Sample output of BenchmarkJS looks like:
<img width="1438" alt="Screenshot 2023-07-31 at 1 29 10 PM"
src="https://github.com/AzureAD/microsoft-authentication-library-for-js/assets/87724641/f9dc302e-84b5-4ca8-820b-106502ab8684">

If a regression of >10% is detected, github actions will leave a commit
comment:
<img width="1882" alt="Screenshot 2023-07-31 at 1 28 40 PM"
src="https://github.com/AzureAD/microsoft-authentication-library-for-js/assets/87724641/9e0a6252-baab-4c53-abee-69068bed3ed3">
[Example of commit comment when a fake regression was introduced to this
PR.](917e9c4#commitcomment-123212353)

Every time a PR is merged into dev, the latest output from
"regression-tests/msal-node/client-credential/output.txt" (if available,
if the client-credential test was run on the PR) will be saved to the
branch "gh-pages", and will be available to visually see at:
[https://azuread.github.io/microsoft-authentication-library-for-js/dev/bench/](https://azuread.github.io/microsoft-authentication-library-for-js/dev/bench/).
  • Loading branch information
Robbie-Microsoft authored Sep 12, 2023
1 parent 2bc6389 commit a82c117
Show file tree
Hide file tree
Showing 6 changed files with 280 additions and 0 deletions.
46 changes: 46 additions & 0 deletions .github/workflows/client-credential-benchmark.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,46 @@
name: msal-node client-credential Regression Test
on:
# Allows manual triggering of workflow
workflow_dispatch:

pull_request:
types: [opened, reopened, synchronize, ready_for_review]
paths:
- "regression-tests/msal-node/client-credential/*"
- "lib/msal-node/**/*"
- "lib/msal-common/**/*"
- "!**.md"
- ".github/workflows/client-credential.yml"

merge_group:
types: [checks_requested]

jobs:
benchmark:
name: Run msal-node client-credential Regression Test
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
- name: Run benchmark
run: cd regression-tests/msal-node/client-credential && npm install && node index.js | tee output.txt

- name: Download previous benchmark data
uses: actions/cache@v3
with:
path: ./cache
key: ${{ runner.os }}-benchmark

- name: Store benchmark result
uses: benchmark-action/github-action-benchmark@v1
with:
name: msal-node client-credential Regression Test
tool: 'benchmarkjs'
output-file-path: regression-tests/msal-node/client-credential/output.txt
external-data-json-path: ./cache/client-credential-benchmark-data.json
github-token: ${{ secrets.GITHUB_TOKEN }}
# Show alert with commit comment on detecting possible performance regression
alert-threshold: '110%'
comment-on-alert: true
fail-on-alert: false
alert-comment-cc-users: '@bgavrilMS @robbie-microsoft'
35 changes: 35 additions & 0 deletions .github/workflows/msal-node-confidential-client-benchmarks.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,35 @@
# Do not run this workflow on pull request since this workflow has permission to modify contents.
on:
push:
branches:
- dev

permissions:
# deployments permission to deploy GitHub pages website
deployments: write
# contents permission to update benchmark contents in gh-pages branch
contents: write

jobs:
benchmark:
name: Performance regression check
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v3
- uses: actions/setup-node@v3
- name: Run benchmark
run: cd regression-tests/msal-node/client-credential && npm install && node index.js | tee output.txt

# gh-pages branch is updated and pushed automatically with extracted benchmark data
# By default, this action assumes that gh-pages is the GitHub Pages branch and that
# /dev/bench is a path to put the benchmark dashboard page. They can be tweaked by
# the gh-pages-branch, gh-repository and benchmark-data-dir-path inputs.
- name: Store benchmark result
uses: benchmark-action/github-action-benchmark@v1
with:
name: msal-node client-credential Regression Test
tool: 'benchmarkjs'
output-file-path: regression-tests/msal-node/client-credential/output.txt
github-token: ${{ secrets.GITHUB_TOKEN }}
# Push and deploy GitHub pages branch automatically
auto-push: true
94 changes: 94 additions & 0 deletions regression-tests/msal-node/Constants.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,94 @@
/*
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License.
*/

const NetworkUtils = class NetworkUtils {
// eslint-disable-next-line @typescript-eslint/explicit-module-boundary-types
static getNetworkResponse(headers, body, statusCode) {
return {
headers: headers,
body: body,
status: statusCode,
};
}
};

const DEFAULT_OPENID_CONFIG_RESPONSE = {
headers: {},
status: 200,
body: {
token_endpoint:
"https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token",
token_endpoint_auth_methods_supported: [
"client_secret_post",
"private_key_jwt",
"client_secret_basic",
],
jwks_uri:
"https://login.microsoftonline.com/{tenant}/discovery/v2.0/keys",
response_modes_supported: ["query", "fragment", "form_post"],
subject_types_supported: ["pairwise"],
id_token_signing_alg_values_supported: ["RS256"],
response_types_supported: [
"code",
"id_token",
"code id_token",
"id_token token",
],
scopes_supported: ["openid", "profile", "email", "offline_access"],
issuer: "https://login.microsoftonline.com/{tenant}/v2.0",
request_uri_parameter_supported: false,
userinfo_endpoint: "https://graph.microsoft.com/oidc/userinfo",
authorization_endpoint:
"https://login.microsoftonline.com/{tenant}/oauth2/v2.0/authorize",
http_logout_supported: true,
frontchannel_logout_supported: true,
end_session_endpoint:
"https://login.microsoftonline.com/{tenant}/oauth2/v2.0/logout",
claims_supported: [
"sub",
"iss",
"cloud_instance_name",
"cloud_instance_host_name",
"cloud_graph_host_name",
"msgraph_host",
"aud",
"exp",
"iat",
"auth_time",
"acr",
"nonce",
"preferred_username",
"name",
"tid",
"ver",
"at_hash",
"c_hash",
"email",
],
tenant_region_scope: null,
cloud_instance_name: "microsoftonline.com",
cloud_graph_host_name: "graph.windows.net",
msgraph_host: "graph.microsoft.com",
rbac_url: "https://pas.windows.net",
},
};

const CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT = {
headers: {},
status: 200,
body: {
token_type: "Bearer",
expires_in: 3599,
ext_expires_in: 3599,
refresh_in: 3598 / 2,
access_token: "thisIs.an.accessT0ken",
},
};

module.exports = {
CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT,
DEFAULT_OPENID_CONFIG_RESPONSE,
NetworkUtils,
};
2 changes: 2 additions & 0 deletions regression-tests/msal-node/client-credential/.gitignore
Original file line number Diff line number Diff line change
@@ -0,0 +1,2 @@
node_modules/
package-lock.json
87 changes: 87 additions & 0 deletions regression-tests/msal-node/client-credential/index.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,87 @@
/*
* Copyright (c) Microsoft Corporation. All rights reserved.
* Licensed under the MIT License.
*/

/* eslint-disable @typescript-eslint/no-unused-vars */

import benchmark from "benchmark";
import * as msal from "@azure/msal-node";
import {
CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT,
DEFAULT_OPENID_CONFIG_RESPONSE,
NetworkUtils,
} from "../Constants.js";

const clientConfig = {
auth: {
clientId: "client_id",
authority: "https://login.microsoftonline.com/tenant_id",
knownAuthorities: ["https://login.microsoftonline.com/tenant_id"],
clientSecret: "client_secret",
},
system: {
networkClient: new class CustomHttpClient {
sendGetRequestAsync(url, options, cancellationToken) {
return new Promise((resolve, reject) => {
const networkResponse = NetworkUtils.getNetworkResponse(
DEFAULT_OPENID_CONFIG_RESPONSE.headers,
DEFAULT_OPENID_CONFIG_RESPONSE.body,
DEFAULT_OPENID_CONFIG_RESPONSE.status,
);
resolve(networkResponse);
});
}
sendPostRequestAsync(url, options) {
return new Promise((resolve, _reject) => {
const networkResponse = NetworkUtils.getNetworkResponse(
CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT.headers,
CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT.body,
CONFIDENTIAL_CLIENT_AUTHENTICATION_RESULT.status,
);
resolve(networkResponse);
});
}
}
},
};

const NUM_CACHE_ITEMS = 10;

const confidentialClientApplication = new msal.ConfidentialClientApplication(clientConfig);
const firstResourceRequest = {
scopes: ["resource-1/.default"],
};
const lastResourceRequest = {
scopes: [`resource-${NUM_CACHE_ITEMS}/.default`],
};

(async () => {
// pre populate the cache
for (let i = 1; i <= NUM_CACHE_ITEMS; i++) {
const request = {
scopes: [`resource-${i}/.default`],
};
await confidentialClientApplication.acquireTokenByClientCredential(request);
}

const suite = new benchmark.Suite();
suite
.add("ConfidentialClientApplication#acquireTokenByClientCredential-fromCache-resourceIsFirstItemInTheCache", {
"fn": async () => {
await confidentialClientApplication.acquireTokenByClientCredential(firstResourceRequest);
},
"minSamples": 150,
})
.add("ConfidentialClientApplication#acquireTokenByClientCredential-fromCache-resourceIsLastItemInTheCache", {
"fn": async () => {
await confidentialClientApplication.acquireTokenByClientCredential(lastResourceRequest);
},
"minSamples": 150,
})
.on("cycle", (event) => {
// eslint-disable-next-line no-console
console.log(String(event.target));
})
.run({ "async": true });
})();
16 changes: 16 additions & 0 deletions regression-tests/msal-node/client-credential/package.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,16 @@
{
"name": "client-credential",
"version": "1.0.0",
"description": "Client Credential Regression Test",
"main": "index.js",
"type": "module",
"scripts": {
"test": "echo \"Error: no test specified\" && exit 1"
},
"author": "",
"license": "MIT",
"dependencies": {
"@azure/msal-node": "^2.0.0",
"benchmark": "^2.1.4"
}
}

0 comments on commit a82c117

Please sign in to comment.