From effd75a727c167e2c00224f7422b10e02a6571cd Mon Sep 17 00:00:00 2001 From: Merlin Beutlberger Date: Tue, 16 Apr 2024 15:09:56 +0200 Subject: [PATCH] docs(PERFORMANCE): Update benchmark results --- docs/PERFORMANCE.md | 162 ++++++++++++++++++++++++++++++++++++-------- 1 file changed, 133 insertions(+), 29 deletions(-) diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md index 4d0c2c62..c2a6a91f 100644 --- a/docs/PERFORMANCE.md +++ b/docs/PERFORMANCE.md @@ -3,30 +3,37 @@ For all benchmarks below, execute `ui5lint` without any parameters. Benchmarks created based on [Benchmarking UI5 Tooling](https://sap.github.io/ui5-tooling/stable/pages/Benchmarking/): ```sh -hyperfine --warmup 1 \ +hyperfine -i --warmup 1 \ 'node $(which ui5lint)' \ --export-markdown ./bench.md ``` -**TODO:** Memory benchmarking +## Small app: `openui5-sample-app` + +_Reference commit: [`c9a0f7c51`](https://github.com/SAP/openui5-sample-app/tree/c9a0f7c51)_ + +**Metadata:** + +* Resources to lint: `17` (31.59 KB) +* Latest Result: `0 problems (0 errors, 0 warnings)` -## Non-JS library, no findings: OpenUI5 `themelib_sap_horizon` +## Non-JS library with no findings: OpenUI5 `themelib_sap_horizon` _Reference commit: [`0efb2cd`](https://github.com/SAP/openui5/tree/0efb2cd89a893b499f40c43c7534240524c5de42/src/themelib_sap_horizon) (tag `1.120.10`)_ **Metadata:** * Resources to lint: `0` -* Result: `0 problems (0 errors, 0 warnings)` +* Latest Result: `0 problems (0 errors, 0 warnings)` -## Small app: `openui5-sample-app` +## Small library: OpenUI5 `sap.ui.testrecorder` -_Reference commit: [`fa6d40d`](https://github.com/SAP/openui5-sample-app/tree/fa6d40d2a3ef7c7bb7c416117a0efc675ec90c65)_ +_Reference commit: [`0efb2cd`](https://github.com/SAP/openui5/tree/0efb2cd89a893b499f40c43c7534240524c5de42/src/sap.ui.testrecorder) (tag `1.120.10`)_ **Metadata:** -* Resources to lint: `16` -* Result: `6 problems (6 errors, 0 warnings)` +* Resources to lint: `68` (0.19 MB) +* Latest Result: `13 problems (13 errors, 0 warnings)` ## Medium library: OpenUI5 `sap.ui.layout` @@ -34,8 +41,8 @@ _Reference commit: [`0efb2cd`](https://github.com/SAP/openui5/tree/0efb2cd89a893 **Metadata:** -* Resources to lint: `497` -* Result: `1308 problems (1308 errors, 0 warnings)` +* Resources to lint: `572` (2.4 MB) +* Latest Result: `1140 problems (1140 errors, 0 warnings)` ## Large library: OpenUI5 `sap.m` @@ -43,34 +50,69 @@ _Reference commit: [`0efb2cd`](https://github.com/SAP/openui5/tree/0efb2cd89a893 **Metadata:** -* Resources to lint: `4914` -* Result: `11937 problems (11937 errors, 0 warnings)` -* This is around 10x more than sap.ui.layout +* Resources to lint: `5609` (25.67 MB) +* Latest Result: `11293 problems (11293 errors, 0 warnings)` + + +## Large library: OpenUI5 `sap.ui.core` + +_Reference commit: [`0efb2cd`](https://github.com/SAP/openui5/tree/0efb2cd89a893b499f40c43c7534240524c5de42/src/sap.ui.core) (tag `1.120.10`)_ + +**Metadata:** + +* Resources to lint: `4950` (45 MB) +* Latest Result: `4816 problems (4816 errors, 0 warnings)` + `2 fatal errors` # Benchmark Runs +## April 17, 2024 + +* UI5 linter [`33d8cc5`](https://github.com/SAP/ui5-linter/commit/33d8cc5) +* Node.js `v20.11.1` +* MacBook Pro M1 Max + +### themelib_sap_horizon + +| Mean [ms] | Min [ms] | Max [ms] | +|---:|---:|---:| +| 595.7 ± 22.8 | 545.1 | 630.6 | + +### openui5-sample-app + +Mean [s] | Min [s] | Max [s] | +|---:|---:|---:| +| 2.242 ± 0.043 | 2.182 | 2.322 | + +### sap.ui.testrecorder + +| Mean [s] | Min [s] | Max [s] | +|---:|---:|---:| +| 3.605 ± 0.071 | 3.520 | 3.768 | + +### sap.ui.layout + +| Mean [s] | Min [s] | Max [s] | +|---:|---:|---:| +| 5.995 ± 0.045 | 5.934 | 6.086 | + +### sap.m + +| Mean [s] | Min [s] | Max [s] | +|---:|---:|---:| +| 38.769 ± 0.508 | 38.096 | 39.585 | + +### sap.ui.core + +| Mean [s] | Min [s] | Max [s] | +|---:|---:|---:| +| 53.864 ± 2.036 | 50.265 | 56.210 | + ## March 11, 2024 * UI5 linter [`5466be5`](https://github.com/SAP/ui5-linter/commit/5466be5b983c4c6e6108c0d97d5221b8ad320a88) * Node.js `v20.11.1` * MacBook Pro M1 Max -```mermaid ---- -config: - xyChart: - xAxis: - labelPadding: 20 - yAxis: - titlePadding: 20 ---- -xychart-beta - title "March 11, 2024" - x-axis "Project (number of files)" ["themelib_sap_horizon (0)", "openui5-sample-app (16)", "sap.ui.layout (497)", "sap.m (4914)"] - y-axis "Runtime (in milliseconds)" - line [442.6, 2171, 6002, 34470] -``` - ### themelib_sap_horizon | Mean [ms] | Min [ms] | Max [ms] | @@ -79,6 +121,8 @@ xychart-beta ### openui5-sample-app +**Based on [`fa6d40d`](https://github.com/SAP/openui5-sample-app/tree/fa6d40d2a3ef7c7bb7c416117a0efc675ec90c65), which still contained linting errors** + Mean [s] | Min [s] | Max [s] | |---:|---:|---:| 2.171 ± 0.072 | 2.076 | 2.315 | @@ -94,3 +138,63 @@ Mean [s] | Min [s] | Max [s] | | Mean [s] | Min [s] | Max [s] | |---:|---:|---:| | 34.470 ± 0.421 | 34.178 | 35.585 | + + +# Helper Scripts + +```sh +# +cd src/themelib_sap_horizon +ui5lint +hyperfine -i --warmup 1 \ +'node $(which ui5lint)' \ +--export-markdown ../../bench-themelib_sap_horizon.md + +cd ../sap.ui.testrecorder +ui5lint +hyperfine -i --warmup 1 \ +'node $(which ui5lint)' \ +--export-markdown ../../bench-sap.ui.testrecorder.md + +cd ../sap.ui.layout +ui5lint +hyperfine -i --warmup 1 \ +'node $(which ui5lint)' \ +--export-markdown ../../bench-sap.ui.layout.md + +cd ../sap.m +ui5lint +hyperfine -i --warmup 1 \ +'node $(which ui5lint)' \ +--export-markdown ../../bench-sap.m.md + +cd ../sap.ui.core +ui5lint +hyperfine -i --warmup 1 \ +'node $(which ui5lint)' \ +--export-markdown ../../bench-sap.ui.core.md +``` + +```sh +# + +ui5lint +hyperfine -i --warmup 1 \ +'node $(which ui5lint)' \ +--export-markdown ./bench.md +``` + +Helper script to determine the count and accumulated size of all files to be linted: + +```js +// For a one-time evaluation, place this for example at the beginning of the lintWorkspace function +// *DO NOT USE THIS WHILE TAKING BENCHMARK MEASUREMENTS* +const allResources = await workspace.byGlob("**/{*.js,*.js.map,*.ts,*.html,manifest.json,manifest.appdescr_variant,*.view.xml,*.fragment.xml,ui5.yaml,*-ui5.yaml,*.ui5.yaml,ui5-*.yaml}") +const sizeOfAllFiles = await allResources.reduce(async (acc, res) => { + const currentLength = await acc + const str = await res.getBuffer(); + return currentLength + str.length; +}, Promise.resolve(0)) +console.log(`Total count of all files to be linted: ${allResources.length}`); +console.log(`Total size of all files: ${sizeOfAllFiles} bytes`); +````