docs(PERFORMANCE): Update benchmark results

SAP · Apr 16, 2024 · 54919e8 · 54919e8
1 parent 508d81d
commit 54919e8
Showing 1 changed file with 127 additions and 19 deletions.
diff --git a/docs/PERFORMANCE.md b/docs/PERFORMANCE.md
@@ -3,55 +3,71 @@
 For all benchmarks below, execute `ui5lint` without any parameters. Benchmarks created based on [Benchmarking UI5 Tooling](https://sap.github.io/ui5-tooling/stable/pages/Benchmarking/):
 
 ```sh
-hyperfine --warmup 1 \
+hyperfine -i --warmup 1 \
 'node $(which ui5lint)' \
 --export-markdown ./bench.md
 ```
 
-**TODO:** Memory benchmarking
+## Small app: `openui5-sample-app`
+
+_Reference commit: [`c9a0f7c51`](https://github.com/SAP/openui5-sample-app/tree/c9a0f7c51)_
+
+**Metadata:**
+
+* Resources to lint: `17` (31.59 KB)
+* Latest Result: `0 problems (0 errors, 0 warnings)`
 
-## Non-JS library, no findings: OpenUI5 `themelib_sap_horizon`
+## Non-JS library with no findings: OpenUI5 `themelib_sap_horizon`
 
 _Reference commit: [`0efb2cd`](https://github.com/SAP/openui5/tree/0efb2cd89a893b499f40c43c7534240524c5de42/src/themelib_sap_horizon) (tag `1.120.10`)_
 
 **Metadata:**
 
 * Resources to lint: `0`
-* Result: `0 problems (0 errors, 0 warnings)`
+* Latest Result: `0 problems (0 errors, 0 warnings)`
 
-## Small app: `openui5-sample-app`
+## Small library: OpenUI5 `sap.ui.testrecorder`
 
-_Reference commit: [`fa6d40d`](https://github.com/SAP/openui5-sample-app/tree/fa6d40d2a3ef7c7bb7c416117a0efc675ec90c65)_
+_Reference commit: [`0efb2cd`](https://github.com/SAP/openui5/tree/0efb2cd89a893b499f40c43c7534240524c5de42/src/sap.ui.testrecorder) (tag `1.120.10`)_
 
 **Metadata:**
 
-* Resources to lint: `16`
-* Result: `6 problems (6 errors, 0 warnings)`
+* Resources to lint: `68` (0.19 MB)
+* Latest Result: `13 problems (13 errors, 0 warnings)`
 
 ## Medium library: OpenUI5 `sap.ui.layout`
 
 _Reference commit: [`0efb2cd`](https://github.com/SAP/openui5/tree/0efb2cd89a893b499f40c43c7534240524c5de42/src/sap.ui.layout) (tag `1.120.10`)_
 
 **Metadata:**
 
-* Resources to lint: `497`
-* Result: `1308 problems (1308 errors, 0 warnings)`
+* Resources to lint: `572` (2.4 MB)
+* Latest Result: `1140 problems (1140 errors, 0 warnings)`
 
 ## Large library: OpenUI5 `sap.m`
 
 _Reference commit: [`0efb2cd`](https://github.com/SAP/openui5/tree/0efb2cd89a893b499f40c43c7534240524c5de42/src/sap.m) (tag `1.120.10`)_
 
 **Metadata:**
 
-* Resources to lint: `4914`
-* Result: `11937 problems (11937 errors, 0 warnings)`
-* This is around 10x more than sap.ui.layout
+* Resources to lint: `5609` (25.67 MB)
+* Latest Result: `11293 problems (11293 errors, 0 warnings)`
+
+
+## Large library: OpenUI5 `sap.ui.core`
+
+_Reference commit: [`0efb2cd`](https://github.com/SAP/openui5/tree/0efb2cd89a893b499f40c43c7534240524c5de42/src/sap.ui.core) (tag `1.120.10`)_
+
+**Metadata:**
+
+* Resources to lint: `4950` (45 MB)
+* Latest Result: `4816 problems (4816 errors, 0 warnings)` + `2 fatal errors`
 
 # Benchmark Runs
 
-## March 11, 2024
+## April 16, 2024
 
-* UI5 linter [`5466be5`](https://github.com/SAP/ui5-linter/commit/5466be5b983c4c6e6108c0d97d5221b8ad320a88)
+* UI5 linter [`6315093`](https://github.com/SAP/ui5-linter/commit/6315093)
 * Node.js `v20.11.1`
 * MacBook Pro M1 Max
 
@@ -65,20 +81,64 @@ config:
             titlePadding: 20
 ---
 xychart-beta
-    title "March 11, 2024"
-    x-axis "Project (number of files)" ["themelib_sap_horizon (0)", "openui5-sample-app (16)", "sap.ui.layout (497)", "sap.m (4914)"]
-    y-axis "Runtime (in milliseconds)" 
-    line [442.6, 2171, 6002, 34470]
+    title "April 16, 2024"
+    x-axis "Project (total file size in byte)" ["horizon (0)", "s. app (0.03 MB)", "testrec. (0.19 MB)", "layout (2.4 MB)", "m (25.67 MB)", "core (45 MB)"]
+    y-axis "Runtime (in seconds)" 
+    line [0.553, 2.184, 2.735, 5.985, 35.241, 51.558]
 ```
 
 ### themelib_sap_horizon
 
+| Mean [ms] | Min [ms] | Max [ms] |
+|---:|---:|---:|
+| 553.0 ± 37.6 | 496.2 | 612.3 |
+
+### openui5-sample-app
+
+Mean [s] | Min [s] | Max [s] |
+|---:|---:|---:|
+| 2.184 ± 0.075 | 2.101 | 2.363 |
+
+### sap.ui.testrecorder
+
+| Mean [s] | Min [s] | Max [s] |
+|---:|---:|---:|
+| 2.735 ± 0.063 | 2.633 | 2.839 |
+
+### sap.ui.layout
+
+| Mean [s] | Min [s] | Max [s] |
+|---:|---:|---:|
+| 5.985 ± 0.052 | 5.909 | 6.064 |
+
+### sap.m
+
+| Mean [s] | Min [s] | Max [s] |
+|---:|---:|---:|
+| 35.241 ± 1.143 | 33.392 | 37.097 |
+
+### sap.ui.core
+
+| Mean [s] | Min [s] | Max [s] |
+|---:|---:|---:|
+| 51.558 ± 0.959 | 50.028 | 53.065 |
+
+## March 11, 2024
+
+* UI5 linter [`5466be5`](https://github.com/SAP/ui5-linter/commit/5466be5b983c4c6e6108c0d97d5221b8ad320a88)
+* Node.js `v20.11.1`
+* MacBook Pro M1 Max
+
+### themelib_sap_horizon
+
 | Mean [ms] | Min [ms] | Max [ms] |
 |---:|---:|---:|
 | 442.6 ± 33.8 | 409.4 | 529.5 |
 
 ### openui5-sample-app
 
+**Based on [`fa6d40d`](https://github.com/SAP/openui5-sample-app/tree/fa6d40d2a3ef7c7bb7c416117a0efc675ec90c65), which still contained linting errors**
+
 Mean [s] | Min [s] | Max [s] |
 |---:|---:|---:|
 2.171 ± 0.072 | 2.076 | 2.315 |
@@ -94,3 +154,51 @@ Mean [s] | Min [s] | Max [s] |
 | Mean [s] | Min [s] | Max [s] |
 |---:|---:|---:|
 | 34.470 ± 0.421 | 34.178 | 35.585 |
+
+
+# Helper Scripts
+
+```sh
+# <Inside the openui5 repository root>
+cd src/themelib_sap_horizon
+ui5lint
+hyperfine -i --warmup 1 \
+'node $(which ui5lint)' \
+--export-markdown ../../bench-themelib_sap_horizon.md
+
+cd ../sap.ui.layout
+ui5lint
+hyperfine -i --warmup 1 \
+'node $(which ui5lint)' \
+--export-markdown ../../bench-sap.ui.layout.md
+
+cd ../sap.m
+ui5lint
+hyperfine -i --warmup 1 \
+'node $(which ui5lint)' \
+--export-markdown ../../bench-sap.m.md
+```
+
+```sh
+# <Inside the openui5-sample-app root>
+
+ui5lint
+hyperfine -i --warmup 1 \
+'node $(which ui5lint)' \
+--export-markdown ./bench.md
+```
+
+Helper script to determine the count and accumulated size of all files to be linted:
+
+```js
+// For a one-time evaluation, place this for example at the beginning of the lintWorkspace function
+// *DO NOT USE THIS WHILE TAKING BENCHMARK MEASUREMENTS*
+const allResources = await workspace.byGlob("**/{*.js,*.js.map,*.ts,*.html,manifest.json,manifest.appdescr_variant,*.view.xml,*.fragment.xml,ui5.yaml,*-ui5.yaml,*.ui5.yaml,ui5-*.yaml}")
+const sizeOfAllFiles = await allResources.reduce(async (acc, res) => {
+	const currentLength = await acc
+	const str = await res.getBuffer();
+	return currentLength + str.length;
+}, Promise.resolve(0))
+console.log(`Total count of all files to be linted: ${allResources.length}`);
+console.log(`Total size of all files: ${sizeOfAllFiles} bytes`);
+````